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GUTOELINES  FOR  DEVELOPING  RADIATION  HARDNESS 
ASSURANCE  DEVICE  SPECIFICATIONS 


1  SCOPE 

1*1  The  primary  oblective  of  this  document  is  to  provide  guidelines  and 
easy  to  follow  procedures  for  the  preparation  of  detailed  device  specifications 
for  the  procurement  of  microcircuits  and  semiconductor  devices  where  radiation 
hardness  assurance  (RHA)  Is  required.  The  guidelines  are  applicable  to  MIL-M- 
38510,  MIL-S— 19500  microcircuit  and  semiconductor  device  detailed  specifications 
as  well  as  to  other  specifications  such  as  source  control  drawings  (SOCD) , 
selected  item  drawings  (SID),  specification  control  drawings  (SCD) ,  and 
standardized  military  drawings  (SMD).  Recommended  procedures  are  provided  for 
characterizing  the  radiation  response  of  a  part  and  for  obtaining  post- 
irradlatlon  end-point  limits  for  qualification  and  lot  acceptance  tests  (LAT) . 

1-2  Radiation  hardness  assurance  at  the  plecenart  level.  These  guidelines 
address  radiation  response  measurement  and  hardness  assurance  questions  at  the 
plecepart  level.  In  keeping  with  the  present  scheme  for  MIL  STD  RHA  device 
specifications,  which  only  addresses  Ionizing  radiation  dose  effects  measured  at 
200,  plus  or  minus  100  rads(Sl)/second  and  displacement  damage  effects  due  to 
neutrons  (see  TABLE  1  of  Section  4.2),  these  guidelines  emphasize  these  two 
radiation  environments.  However,  because  the  addition  of  dose  rate  and  SEU 
specifications  is  now  under  consideration,  brief  mention  of  these  two  radiation 
environments  is  also  included.  The  principles  discussed  are  applicable  to  both 
bipolar  and  MOS. silicon  transistors  and  Integrated  circuits,  and  to  devices  made 
from  gallium  arsenide  and  other  semiconductor  materials.  They  have,  furthermore, 
been  presented  in  terms  of  the  intrinsic  performance  characteristics  of  the  part 
independent  of  any  system  in  which  it  may  be  used.  For  MIL  STD  specifications , 
the  levels  shown  in  TABLE  1  of  Section  4.2  are  used  as  reference  points.  The 
general  principles  may  be  applied  to  system  specific  requirements. 

1*3  LAT  end-polnt  limits.  The  procedures  for  measuring  radiation  response 
characteristics  and  for  calculating  LAT  end-polnt  limits  are  discussed  in  terms 
of  parts  whose  design  and  production  processes  are  mature.  That  is,  these 
guidelines  do  not  attempt  to  discuss  the  case  where  the  part  is  still  under 
development  and  its  characteristics  are  still  undergoing  change.  It  is 
recognized  that  this  latter  case  is  Important  and  occurs  frequently  because 
system  designers  are  interested  in  obtaining  the  most  advanced  parts  for  their 
systems.  The  process  of  obtaining  LAT  end-point  limits  for  such  parts,  however, 
involves  testing  and  iterative  end-point  adjustments  by  the  part  manufacturer  and 
the  system  parts  engineers  and  designers  which  would  be  difficult  to  formulate 
as  a  set  of  generalized  steps  which  could  be  applied  to  a  variety  of  system 
needs.  Because  MIL  STD  procurements  of  RHA  devices  use  attribute  (LTPD)  LAT 
tests  exclusively,  the  end-point  limit  discussions  here  emphasize  LTPD  tests. 

1.4  Low  yield  devices.  If  sample  costs  are  affordable,  the  LAT  methods 
discussed  will  be  performed  on  samples  of  the  devices  themselves.  In  the  case  of 
very  large  scale  Integration  (VLSI)  devices  for  which  only  low  yields  can  be 
achieved,  one  possible  option  might  be  to  use  devices  for  LAT  which  are 
acceptable  from  an  electrical  performance  standpoint  but  do  not  meet  all  the 
normal  visual  acceptance  criteria.  Such  devices  are  said  to  be  selected 
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according  to  "alternate”  visual  criteria.  Another  possible  option  might  be  to 
use  test  structures  for  LAT  which  have  been  processed  on  the  same  wafers  as  the 
lot  under  consideration.  By  test  structures  are  meant  simpler  and  less  expensive 
microcircuits  which  have  been  designed  specifically  to  correlate  with  the 
radiation  response  characteristics  and  failure  levels  of  the  subject  VLSI 
circuit;  radiation  tests  on  the  test  structure  can  then  be  used  to  estimate  the 
performance  characteristics  of  the  VLSI  circuit.  The  use  of  test  structures  for 
LAT  cannot  be  recommended  until  data  and  experience  show  definitively  that  test 
structure  responses  correlate  reliably  with  the  actual  devices. 

1.5  Generic  Qualification  nroeram.  A  fundamentally  different  approach  to 
quality  assurance,  from  that  based  mostly  on  testing  the  end  product,  is  now 
being  implemented.  An  overly  simplified  description  of  this  approach  is  to  say 
that  it  is  based  on  tests  which  will  continuously  measure  the  quality  of  the 
starting  materials  and  of  the  production  process  Itself  and  allow  the  material 
purity  and  the  process  to  be  controlled  so  that  overall  quality  is  not  only 
maintained  but  is  Improved  with  time.  This  approach,  which  uses  a  qualified 
manufacturers  list  (QML) ,  is  not  yet  in  place  and  the  way  in  which  hardness 
assurance  will  be  achieved  under  it  has  not  yet  been  determined.  With  respect 
to  the  military  standard  procurement  or  "JAN”  system  this  approach  is  also  termed 
"Generic  Qtialiflcation"  (GQ)  because  it  will  allow  a  vendor  to  qualify  a 
particular  production  line  and  then  to  ship  a  variety  of  part  types  from  that 
line  without  having  to  qualify  each  new  part  type  separately  (as  is  required 
vinder  the  present  q\ialified  parts  list  or  QPL  scheme).  GQ  will  be  important 
because  it  is  expected  to  improve  radiation  response  uniformity  over  extended 
periods  of  time.  FIGURE  1  shows  some  of  the  features  of  the  proposed  generic 
qualification  system.  A  new  draft  controlling  document  has  recently  been  issued 
by  RADC  \diich  is  entitled:  MIL-I-38535,  "General  Specifications  for  Integrated 
Circuit  Manufacturing."  This  document  includes  RHA  requirements  in  all  the 
appropriate  sections  but  does  so  only  in  general  terms.  A  "strawman"  plan 
entitled:  "Methodology  for  Including  Radiation  Hardness  Assurance  in  the  Generic 
qualification  Program,"  is  presently  undergoing  review. 
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FIGURE  1.  Proposed  generic  qualification  system. 


3 


2  APPLICABLE  DOCUMENTS 


Although  these  Guidelines  are  intended  to  be  used  as  a  "stand  alone" 
document,  the  following  reports  and  publications  may  be  found  useful. 

2.1  Government  Documents. 

2.1.1  _ standards,  and  handbooks.  The  following 

specifications,  standards  and  handbooks  form  a  part  of  this  docvunent  to  the 
extent  specified  herein.  Unless  otherwise  specified,  the  issues  of  these 
dociunents  are  those  listed  in  the  issue  of  the  Department  of  Defense  Index  of 
Specifications  and  Standards  (DODISS)  and  supplement  thereto,  cited  in  the 
solicitation. 

SPECIFICATIONS 

MILITARY 

DNA-TR-84-220-V1 ;  Hardness  Assured  Device  Specifications  for 
Moderate  Requirements,  I.  Arimura,  L.  Kolb,  and  0.  R.  Mulkey,  18 
November  1985 

DNA-TR-84-220-V2 ;  Hardness  Assured  Device  Specification  a  4K  X  1 
CMOS/SOS  Static  RAM,  I.  Arimura,  R.  Kennerud,  and  L.  Kolb,  11 
February  1985 

DNA— TR— 81— 90;  Hardness  Assured  Device  Specifications,  I.  Arimura, 
R.  Kennerud,  and  0.  Mulkey,  15  July  1982 

STANDARDS 

MILITARY 

MIL-S-19500;  Semiconductor  Devices,  General  Specification  for, 
MIL-M— 38510;  Microcircuits,  General  Military  Specification  for, 
MIIr-STD— 750;  Test  Methods  for  Semiconductor  Devices 
HII^STD-883;  Test  Methods  and  Procediures  for  Microelectronics 
HANDBOOK 
MILITARY 

DNA  5909F;  Total— Dose  Hardness  Assurance  Guidelines  for 
Semiconductor  Devices,  W.  E.  Price,  K.  E.  Martin,  and  M.  K. 
Gauthier,  1  February  1982. 
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DNA  5910F;  Piece  Part  Neutron  Hardness  Assurance  Guidelines  for 
Semiconductor  Devices,  A.  Namenson,  E.  Wolicki,  R.  Berger,  H. 
Eisen,  J.  Ferry,  G.  Messenger,  R.  Scace,  and  H.  Schafft,  6  October 
1981. 

MIL— HDBK— 279;  Total  Dose  Hardness  Assurance  Guidelines  for 
Semiconductor  Devices  and  Microcircuits,  25  January  1985 

MIL-HDBK-280;  Neutron  Hardness  Assurance  Guidelines  for 
Semiconductor  Devices  and  Microcircuits,  19  February  1985 

MIL— HDBK— 339  (USAF) ;  Custom  Large  Scale  Integrated  Circuit 
Development  and  Acquisition  for  Space  Vehicles.  31  July  1984 

MIL-HDBK-814:  Total  Dose  and  Neutron  Hardness  Assurance  Guidelines 
for  Semiconductor  Devices  and  Microcircuits,  J.  Ferry,  AFWL,  to, 
be  published. 

AFWL— TR— 86— 26 ;  Guidelines  to  Hardness  Assurance  for  Nuclear 
Radiation,  Blast  and  Thermal  Effects  in  Systems  with  Moderate 
Requirements,  J.  M.  Ferry,  February  1987 

DNA-TR-86-29 ;  Dose  Rate  Hardness  Assurance  Guidelines,  J. 
Azarewicz,  14  November  1985 

DNA-TR-86-38 ;  Hardness  Assurance  Guideline  for  MIL-HDBK-339  (USAF) 
Custom  Large  Scale  Integrated  Circuit  development  and  Acquisition 
for  Space  Vehicles,  R.  L.  Pease  and  J.  P.  Raymond,  27  January  1986 

HDLr-CR-81-015-1 ;  Design  Guidelines  for  Transient  Radiation  Effects 
on  Tactical  Amqr  Systems,  M.  Rose,  J.  Harrity,  N.  Rudie,  K 
Rasmussen,  and  J.  Halpin,  July  1981 

HDL-SR— 86— 1;  Nuclear  Hardness  Assurance  and  Hardness  Maintenance 
for  Army  Tactical  systems:  Part  1.  Initial  Nuclear  radiation. 
Thermal  Radiation,  and  Air  Blast,  P.  A  Trimmer,  C.  E,  Spyropoulos, 
and  L.  J.  Belllveau 

2.1.2  Other  Government  documents,  drawings,  and  publications.  The 
following  other  Government  publication  forms  a  part  of  this  document  to  the 
extent  specified  herein. 

PUBLICATIONS 

FEDERAL 

SCR-607;  Factors  for  One-Sided  Tolerance  Limits  and  for  Variables 
Sampling  Plans,  D.  B.  Owen,  Sandia  Corp.,  March  1963 
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(Available  fromU.S.  Department  of  Commerce,  National  Technical  Information 
Service. ) 


Experimental  Statistics,  M.  G.  Natrella,  National  Bureau  of 
Standards,  Handbook  91  (1966) 

(Available  from  the  U.S.  Government  Printing  Office,  Washington,  DC  20402) 

2.2  Non-Government  publications.  The  following  documents  form  a  part  of 
this  document  to  the  extent  specified  herein. 

AMERICAN  SOCIETY  FOR  TESTING  AND  MATERIALS  (ASTM) 

1989  Annual  Book  of  ASTM  Standards,  Volume  12.02,  Nuclear,  Solar, 
and  Geothermal. 

1989  Annvial  Book  of  ASTM  standards,  Volume  10.04,  Electronics. 

(Application  for  copies  should  be  addressed  to  the  American  Society  for 
Testing  and  Materials,  1916  Race  Street,  Philadelphia,  PA  19103-1187.) 

Statistical  Treatment  of  Damage  factors  for  semiconductor  devices,  A.  I. 
Namenson,  IEEE  Trans.  Nucl.  Sci. ,  NS-26,  4691  (1979) 

Hardness  Assxirance  and.  Overtesting,  A.  I.  Namenson,  IEEE  Trans.  Nucl. 
Sci.,  NS-29,  1821  (1982) 

Hardness  Assurance  Statistical  Methodology  for  Semiconductor  Devices ,  I . 
Arimura  and  A.  I.  Namenson,  IEEE  Trans.  Nucl.  Sci.,  NS-30,  4322  (1983) 

Proton  Upsets  in  Orbit,  W.  L.  Bendel  and  E.  L.  Petersen,  IEEE  Trans. 
Nucl.  Sci.,  NS-30,  4481  (1983) 

Statistical  Analysis  of  Step  Stress  Measurements  in  Hardness  Assurance, 
A.  I.  Namenson,  IEEE  Trans.  Nucl.  Sci.,  NS-31,  1398  (1984) 

Estimating  Electrical  Parameter  End  Points  for  Devices  Which  Suffer 
Abrupt  Functional  Failure  During  Radiation  testing,  A.  Namenson  and  I 
Arimura,  IEEE  Trans.  Nucl.  Sci.,  NS-32,  4250  (1985) 

A  Logical  Methodology  for  Determining  Electrical  End  Points  for  Multiloc 
and  Multiparameter  Data,  A.  Namenson  and  I.  Arimura,  IEEE  Trans.  Nucl. 
Sci.,  NS-34,  1726  (1987) 

The  Effects  of  Radiation  on  Electronic  Systems,  G.  Messenger  and  M.  Ash, 
Van  Nostrand  Rheinhold,  1987. 

Ionizing  Radiation  Effects  in  MOS  Devices  and  Circuits,  T.  P.  Ma  and  P. 
V.  Dressendorfer,  Wiley,  New  York,  1989. 
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3.  ACRONYMS,  DEFINITIONS,  AND  SYMBOLS 

3.1  Acronyms  used  In  this  standard.  The  acronyms  used  in  this  standard  are 
e  defined  as  follows: 


a. 

ASTM 

— 

American  Society  for  Testing  and  Materials 

b. 

DESC 

Defense  Electronic  Supply  Center 

c. 

DNA 

- 

Defense  Nuclear  Agency 

d. 

DOD 

- 

Department  of  Defense 

e. 

EFL 

- 

End-Point  Limits 

f- 

ERRIC 

- 

Electronics  lUtdiation  Response  Information 

Center 

g* 

ESD 

- 

Electrostatic  Discharge 

h. 

GQ 

- 

Generic  Qualification 

i. 

HA 

- 

Hardness  Assurance 

j. 

JAN 

- 

Joint  Army  Navy  (military  standard) 

k. 

LAT 

- 

Lot  Acceptance  Tests 

1. 

LET 

- 

Linear  Energy  Transfer  produced  when  a 
ionizing  particle  traverses  a  solid 

high  energy 

m. 

LTPD 

- 

Lot  Tolerance  Percent  Defective 

n. 

MIL  STD 

- 

Military  Standard 

o. 

MIL  HDBK 

- 

Military  Handbook 

P- 

MOS 

- 

Metal  Oxide  Semiconductor 

q- 

NASA 

- 

National  Aeronautics  and  Space  Administration 

r. 

SPAWAR 

- 

Space  and  Naval  Warfare  Systems  Command 

s. 

PIPL 

- 

Post-Irradiation  Parameter  Limit,  the  same 
this  document 

as  RHEPL  in 

t. 

PM 

- 

Process  Monitor 

u. 

QA 

— 

Quality  Assurance 
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V.  QCI 


Quality  Conformance  Inspection 

w.  QM  -  Quality  Management 

X.  QML  -  Qualified  Manufacturer's  List 

y.  QFL  -  Qualified  Parts  List 

z.  QRA  -  Quality  and  Reliability  Assurance 

aa.  RADC  -  Rome  Air  Development  Center 

ab.  RHA  -  Radiation  Hardness  Assured  or  Radiation  Hardness 

Assurance 

ac.  RHEFL  -  Radiation  Hardness  Assurance  End-Point  Limit 

ad.  SCD  -  Specification  Control  Drawing 

ae.  SEC  -  Standard  Evaluation  Circuit 

af.  SEP  —  Single  Event  Phenomena 

ag.  SEU  —  Single  Event  Upset 

ah.  SID  -  Selected  Item  Drawing 

al.  SMD  -  Standardized  Military  Drawing 

aj .  SOCD  -  Source  Control  Drawing 

ak.  SPC  —  Statistical  Process  Control 

al.  STD  -  Standard 


am.  TCI  —  Technology  Conformance  Inspections 

an.  TCV  —  Technology  Characterization  Vehicle 

ao.  VLSI  -  Very  Large  Scale  Integration 

3.2  General  usage  for  definitions  and  symbols.  To  make  this  report  easier 
to  read  for  those  who  will  need  to  refer  to  It  only  Infrequently,  an  attempt  has 
been  made  to  devise  a  mathematical  notation  which  Is  more  self-explanatory  than 
the  notations  used  In  other  recent  reports  related  to  hardness  assurance.  This 
notation  Is  used  In  the  definitions  listed  In  this  section  and  throughout  the 
rest  of  the  report.  As  an  example  of  the  differences,  the  present  notation  uses 
MEAN(PAR)  and  GMEAN(PAR)  for  the  measured  arithmetic  and  geometric  mean  values, 
respectively,  of  a  parameter;  in  several  other  reports  an  overline  on  the  word 
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PAR  is  used  to  indicate  the  arithmetic  mean  value  and  PABj,  is  used  to  denote  the 
geometric  mean  value.  Conversions  to  the  other  notations,  if  necessary,  should 
present  no  difficulties.  The  following  is  a  list  of  specific  definitions: 

^•3  ABSORBED  DOSE:  the  absorbed  energy  usually  expressed  in  rads. 

3. A  c flower  case):  the  acceptance  number  in  an  LTPD  test. 

3.5  CONFIDENCE  LEVEL:  the  chance  of  rejecting  a  lot  where  there  is 

less  than  probability  P  that  any  part  from  the 
lot  can  pass  the  test  conditions. 

3.6  DEVICE  SPECIFICATION:  the  contractual  document  to  which  an 

electronic  device  is  either  sold  or  purchased. 


3.7  DIFFUSION  LOT: 


a  set  of  wafers  heated  at  one  time  in  one 
diffusion  furnace 


3.8  DOSE: 

3.9  DOSE  RATE: 

3.10  EXPfXV: 


3.11  FLOENCE: 

3.12  GOODNESS  OF  FIT: 


3.13  GRAY: 


see  Ionizing  Radiation  Dose. 

rads  per  second  due  to  ionizing  radiation. 

Napier's  constant,  e,  to  the  X-th  power. 
Please  note  that  an  exponent  can  only  be  a 
pure  number .  Therefore ,  if  the  notation  shows 
a  variable  that  has  a  dimension  (such  as  an 
ionizing  radiation  dose  expressed  in  rads,  for 
example)  as  an  exponent,  it  should  be 
understood  that  the  exponent  has  been  made 
dimensionless  by  dividing  it  by  a  quantity 
having  a  value  of  one  but  with  the  same 
dimensions  as  the  variable  in  question. 

the  accumulated  nuiid>er  of  irradiating 
particles  per  square  centimeter. 

the  degree  to  which  a  measured  probability 
distribution  fits  the  assumed  distribution; 
the  CHI-square  test  is  commonly  used  to 
determine  the  goodness  of  fit. 

10,000  ergs  of  absorbed  energy  per  gram; 
equals  100  rads. 


3-lA  i( lower  case): 


the  subscript  used  to  label  the  i-th  device  in 
a  sample  of  devices. 


3-15  I (upper  case): 


the  subscript  used  to  label  the  I-th  lot  in  a 
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sample  of  lots. 


3.16  IN-FLUX:  measurements  made  on  a  device  while  it  is 

being  irradiated. 

3.17  IONIZING  RADIATION  DOSE: 

the  accumulated  absorbed  energy  in  rads  or 
Grays  due  to  ionizing  radiation. 

3.18  JAN  parts:  JAN  parts  are  listed  in  the  Qualified  Products 

List  (QFL)  and  undergo  scheduled  periodic 
audits  by  the  qualifying  activity  to  MIL- 
M38510  and  MIL-STD-883  requirements.  The  JAN 
part  takes  precedence  over  the  Standardized 
Military  Drawing  (SMD)  part. 

3.19  Ktt  (n.C.P) :  One-sided  tolerance  limit.  This  factor  takes 

into  account  the  uncertainties  resulting  from 
small  sample  size  statistics.  It  is  applied 
to  normal  distributions  as  follows:  For  a 
sample  size  n,  if  MEAN(PAR)  and  STDEV(PAR)  are 
the  measured  mean  and  standard  deviation 
respectively  for  parameter  PAR,  then  with 
confidence  C,  there  is  a  probability  P,  that 
future  measurements  of  the  parameter  PAR  will 
be  less  than: 


MEAN(PAR)  +  Kn,(n,  C,  P)  *  STDEV(PAR) 
or  larger  than: 


MEAN(PAR)  -  Kn,(n,  C,  P)  *  STDEV(PAR) 

3.20  LNfX) :  the  natural  logarithm  of  X.  Please  note  that 

the  argument  of  a  logarithm  can  only  be  a  pure 
number.  Therefore,  if  the  notation  shows  a 
logarithm  of  a  variable  that  has  a  dimension 
(such  as  an  ionizing  radiation  dose  expressed 
in  rads,  for  example),  it  should  be  understood 
that  the  argument  has  been  made  dimensionless 
by  dividing  it  by  a  quantity  having  a  value  of 
one  but  with  the  same  dimensions  as  the 
variable  in  question. 


3.21  LOT: 


the  population  of  parts  from  which  a  sample 
has  been  taken. 


3.22  LOT  ACCEPTANCE  TEST:  the  testing  of  a  sample  of  parts  from  a  lot  to 

determine  whether  the  lot  is  acceptable  or 
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not. 


3.23  MEAN(X) : 


the  arithmetic  mean  value  of  X. 


MEAN  (X)-l  T  X. 

n  i4-i  " 


e.g.  MEAN  [PAR  (RAD)]-! 

n 


PAR  i  (RAD) 


e.g.  MEAN  [LN  (PAR  (RAD))]-! 

n 


£  LN  (PAR, (RAD  )) 
1-1 


3.24  GMEANfX) : 


the  geometric  (or  logarithmic)  mean  value  of 
X. 


GMEAN(X)  -  EXP(MEAN(LN(X)) 


e.g.GMEAN(PAR(RAD))  -  EXP(MEAN(LN(PAR(RAD) ) ) 


3.25 

n( lower  case): 

3.26 

N^UDoer  case): 

3.27 

NON-RHA  DEVICES: 

3.28 

STANDARD  DEVICES: 

the  nuatber  of  devices  in  a  sample  of  devices. 

the  number  of  lots  in  a  sample  of  lots. 

parts  that  have  not  been  tested  with  radiation 
or  parts  that  fall  the  radiation  test 
criteria. 

parts  that  meet  the  criteria  for  military 
standard  (MIL  STD  or  JAN)  specifications  or 
for  standardized  military  drawings  (SMD). 


3.29  NON-STANDARD  DEVICES:  parts  that  do  not  meet  the  criteria  for 

military  standard  (MIL  STD  or  JAN) 
specifications  or  for  standardized  military 
drawings  (SMD).  There  are  three  types  of  non¬ 
standard  parts  that  are  used  for  military 
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procurement.  They  are  as  follows:  a)  selected 
Item  drawings  (SID),  b)  source  control 
drawings  (SOCD),  and  c)  specification  control 
drawings  (SCD) . 

3.30  NOT-IN-FLUX:  measurements  made  on  a  device  when  it  is  not 

being  irradiated. 

3.31  OSTL:  one  sided  tolerance  limit. 


3.32  P (upper  case):  the  probability  that  any  part  from  a  lot  can 

pass  the  test  conditions;  also  called  the 
piecepart  survival  probability. 

3.33  PAR^ :  the  pre— rad  parameter  value  measured  for  the 

i-th  device. 


3.34  fMilBACI: 


the  post-rad  parameter  value  measured  for  the 
i-th  device. 


3.35  PHI (SPEC): 

3.36  RAD: 


the  radiation  level  required  in  the  device 
specification  for  lot  acceptance  tests. 

equals  100  ergs  of  absorbed  energy  per  gram. 


3.37  RADIATION  HARDNESS  ASSURANCE: 

the  application  of  methods  and  procedures 
during  the  procurement  of  an  electronic  system 
to  ensure  that  the  radiation  response  of  the 
system  is  within  known  and  acceptable  limits. 


3.38  RADIATION  HARDNESS  MAINTENANCE; 

procedures  applied  during  the  deployment  phase 
to  ensure  that  the  system's  operational 
procedures ,  maintenance  requirements  and  aging 
characteristics  maintain  the  system's 
hardness.  These  procedures  include  tests  and 
inspections . 


3.39  RADIATION  HARDNESS  SURVEILLANCE: 

periodic  inspection  and  testing  during  the 
lifetime  of  deployed  systems  to  ensure  that, 
within  acceptable  tolerances,  the  radiation 
responses  of  the  systems  remain  adequate  for 
mission  completion. 


3.40 


parts  labelled  by  letter  designators  which 
indicate  that  the  part  type  has  passed  lot 
acceptance  tests  based  on  a  sample  of  devices 


tested  to  the  device  specification  radiation 
levels . 

3.41  SELECTED  ITEM  DRAWING:  a  specification  that  is  written  against  any 

standard  part  when  the  system  design  has  a 
critical  requirement  not  covered  by  the 
standard  part  specification  (JAN  or  SMD) .  The 
acronym  for  this  term  is  SID. 

3.42  STANDARDIZED  MILITARY  DRAWING: 

a  specification  written  for  parts  which  are 
listed  in  MIL-BUL-103  and  for  which  DESC  has  a 
certificate  of  compliance  to  MIL-STD  883. 
Approved  sources  of  supply  are  listed  on  this 
drawing.  Periodic  compliance  verification 
audits  are  performed  on  these  vendors  by  the 
qualifying  activity  as  scheduling  allows. 
When  a  vendor  receives  JAN  qualification  on 
this  part,  the  part  is  inactivated  for  new 
design  and  the  JAN  part  takes  precedence. 

3.43  SOURCE  CONTROL  DRAWING:  a  specification  that  is  written  when  a 

specific  source  of  supply  is  required  or  the 
part  is  not  covered  by  a  standard  part 
specification  (JAN  or  SMD) .  The  acronym  for 
this  term  is  SOCD. 


3.44  SPECIFICATION  CONTROL  DRAWING; 

a  specification  written  if  a  part  is  not 
covered  by  a  standard  specification  (JAN  or 
SMD)  and  the  system  does  not  have  any 
requirements  above  the  vendor's  standard  part. 
The  acronym  for  this  term  is  SCD. 

3.45  STANDARD  DEVICES:  parts  that  meet  the  criteria  for  military 

standard  (MIL  STD  or  JAN)  specifications  or 
for  standardized  military  drawings  (SMD) . 

3.46  STDEV(X') :  the  standard  deviation  of  X. 


STDEV  (X) 


1 

n-1 


1  X/2 


£Jx,-mean  (X)]2 


e.g. 


STDEV  C  PAR  (RAD)  ) 


1 

n-1 


[par  1  (RAD) -MEAN  (PAR  (RAD)) 
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e.g.  STDEV  [  LN  (PAR  (RAD))] 


*1  r 

-i-  y  [lN  (  par  (RAD) ) -mean  (LN  (PAR  (RAD  ) 

n-1  i.i 


3.47  STEP-STRESS  TESTING:  a  series  of  discrete  cumulative  Irradiations 

with  measurements  made  after  each  Irradiation 
Increment . 

3.48  TOTAL  DOSE:  the  acctuDulated  absorbed  energy  In  rads  or 

Grays  due  to  Ionizing  radiation;  not 
sufficient  to  determine  device  response  to 
total  Ionizing  radiation  exposure  In  many 
types  of  devices  because  of  time  dependent 
effects. 

3.49  WAFER  LOT:  a  set  of  wafers  made  at  the  same  time  and  with 

the  same  equipment  for  each  processing  step. 
The  MIL-M  38510  definition  Is:  a  wafer  lot 
consists  of  microcircuit  wafers  formed  Into  a 
lot  at  the  start  of  wafer  fabrication  for 
homogeneous  processing  as  a  group,  and 
assigned  a  xmlque  Identifier  or  code  to 
provide  traceability,  and  maintain  lot 
Integrity  throughout  the  fabrication  process. 
MIL-M-38510  allows  parallel  processing  of 
portions  of  the  wafer  lot  through  multiple 
machines  or  process  stations  on  the  same 
certified  line  provided  statistical  quality 
control  assures  and  demonstrates  correlations 
between  stations  and  separately  processed 
portions  of  the  wafer  lot. 
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4.  GENERAL  CONSIDERATIONS 


Device  performance  in  the  radiation  environment.  Many  military  and 
civilian  electronic  systems  must  operate  reliably  in  radiation  environments.  The 
design  of  such  systems  therefore  requires  that  the  performance  of  the  individual 
component  semiconductor  devices  in  radiation  environments  be  characterized.  The 
production  of  the  systems  then  further  requires  that  the  parts  purchased  for 
production  have  characteristics  at  least  as  good  as  those  on  which  the  design  was 
based.  These  guidelines  disctiss  how  to  characterize  the  performance  of 
semiconductor  devices  in  radiation  environments  and  how  to  obtain  hardness 
assurance  through  device  procurement  specifications  based  on  the  characterization 
data.  Applications  to  an  existing  system  of  military  standard  (MIL  STD  or  JAN) 
detailed  specifications  for  radiation  hardness  assured  (RHA)  devices  are 
emphasized  but  the  discussions  may  be  applied  also  to  other  part  specifications. 
Radiation  hardness  assurance  is  generally  achieved  through  the  use  of  lot 
acceptance  tests  (LAT)  in  which  parts  from  the  lot  being  purchased  are  irradiated 
and  tested.  Because  the  MIL  STD  lot  acceptance  tests  rely  largely  on  pass-fail 
tests  in  which  a  part's  performance  is  compared  to  a  specified  end-point  limit, 
these  guidelines  discuss,  specifically,  how  radiation  hardness  assurance  end¬ 
point  limits  (RHEFL)  for  such  tests  should  be  calculated  from  the 
characterization  data. 

4.1.1  Degrading  effects  of  radiation.  The  performance  of  semiconductor 
devices  can  be  degraded  by  exposure  to  radiation  which  produces  ionization  or 
displacement  damage.  Depending  on  the  type  of  radiation  incident  on  the  device, 
the  rate  at  which  energy  is  absorbed,  and  the  total  accumulated  amount  of 
absorbed  energy ,  the  device  performance  can  be  degraded  permanently  or 
temporarily.  Examples  of  temporary  degradation  are:  (a)  circuit  upset  due  to 
the  instantaneous  photocurrent  produced  throughout  the  device  by  a  short  pulse 
of  high  intensity  ionizing  radiation  or  (b)  circuit  upset  due  to  the  collection 
of  charge  from  a  single,  local,  ionized  particle  track  passing  through  the 
device.  Upsets  due  to  photocurrents  are  commonly  called  dose  rate  upsets;  upsets 
due  to  the  collection  of  charge  from  single  ionized  tracks  are  called  single 
event  upsets  (SEUs).  Examples  of  permanent  damage  are:  (a)  device  failures  due 
to  atomic  displacements  produced  in  the  semiconductor  (usiially  by  neutrons) ,  (b) 
failures  due  Co  crapped  charge  in  insulating  layers,  which  are  present  in  most 
devices,  produced  by  ionizing  radiation,  or  (c)  latchup  or  burnout  produced  by 
photoctxrrenCs  or  by  single  ionized  tracks.  Trapped  charge  effects  are  sometimes 
called  ionizing  radiation  dose  effects  or  total  dose  effects.  However,  because 
of  time  dependent  effects  after  irradiation,  specifying  the  total  dose  is  not 
considered  adequate  for  describing  these  effects. 

^•1-2  Rfldiati9n _ Hardness  Assurance .  If  nondestructive  electrical 

measurements  alone  could  be  used  to  predict  the  performance  of  a  part  type  in  a 
radiation  environment,  Chen  Che  hardness  assurance  problem  would  be  largely 
solved.  One  of  the  cases  where  such  measurements  have  proven  useful  has  been 
that  of  using  transistor  current  gain-bandwidth  product,  f^,  measurements  to 
screen  out  (by  requiring  f^  to  be  above  some  minimum  value)  parts  which  would  be 
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extra  sensitive  to  neutron  displacement  damage.  For  Ionizing  radiation  dose,  It 
Is  necessary  to  use  actual  radiation  tests  to  assure  the  radiation  response  of 
a  part.  In  practice,  radiation  hardness  assurance  Is  achieved  most  commonly 
through  the  use  of  lot  acceptance  tests  (LAT)  which  are  performed  on  the  lots 
that  are  being  purchased. 

4.1.3  Dose  rate  tests.  For  the  dose  rate  upsets  discussed  previously,  if 
sufficient  care  Is  used,  radiation  tests  can  be  non-damaging.  Thus  parts  which 
have  been  tested  can  be  used  for  production.  In  this  case  100  percent  of  the 
parts  to  be  used  for  system  production  can  be  tested  (screened)  and  a  high  degree 
of  assurance  can  be  obtained  that  the  parts  will  meet  system  requirements. 
Caution  Is  still  required,  because  there  exists  some  possibility,  especially  for 
complex  microcircuits,  that  the  screening  tests  conducted  will  not  have  covered 
the  worst  case  conditions  or  exercised  all  possible  paths.  Report  number  ONA-TR- 
86-29,  14  November  1985,  entitled:  "Dose  Rate  Hardness  Assurance  Guidelines,"  may 
be  consulted  for  a  detailed  discussion  of  this  problem. 

4.1.4  Displacement  damage,  steady  state  Ionization  and  single  event  upsets. 
For  displacement  damage,  steady  state  Ionization  effects,  and  single  event 
upsets,  the  radiation  tests  are  damaging  and  It  is  generally  not  possible  to  use 
the  tested  parts  in  the  system.  The  use  of  damaging  radiation  tests  thus 
produces  the  central  problem  of  hardness  assurance,  namely,  that  estimates  of  the 
survival  probability  of  the  parts  to  be  used  In  the  system  must  be  based  on 
statistical  inference  from  a  tested  sample  of  parts  which  are  degraded  by  the 
tests  and  therefore  cannot  be  used  In  the  system.  This  problem  Is  compounded  by 
the  fact  that  production  processes  can  vary  so  that  the  currently  produced 
radiation  hard  parts  may  not  be  typical  of  those  that  were  measured  In  the  past. 
This  case,  which  Is  common,  has  been  carefully  reviewed  by  both  users  and 
manufacturers  of  semiconductor  devices.  The  consensus  opinion  Is  that.  If  no 
additional  Information  Is  available,  the  degree  of  hardness  assurance  obtained 
from  the  detailed  specification  Is  limited  statistically  to  that  provided  by  the 
radiation  lot  acceptance  tests  (LAT)  or  quality  conformance  Inspections  (QCI) 
that  are  performed  (the  term  QCI  is  reserved  for  tests  performed  by  the 
manufacturer  before  the  lot  is  approved  for  shipment) . 

4.2  MIL  STP  1?r<?CW9Bgnt  gystgn-  in  order  to  make  high  reliability  hardness 
assured  devices  available  to  designers  and  manufacturers  of  systems,  the  existing 
MIL  STD  procurement  system  was  augmented  a  few  years  ago  to  Include  radiation 
hardness  assured  (RHA)  semiconductor  devices .  At  the  present  time ,  JAN  RHA  parts 
are  labelled  by  letter  designators  which  Indicate  that  the  part  type  has  passed 
lot  acceptance  tests  based  on  a  sample  of  parts  tested  at  the  radiation  levels 
shown  In  TABLE  1. 
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TABLE  1.  Military  Standard  RHA  detailed  specifications. 


LETTER  DESIGNATOR* 

/ 

M 

D 

R 

H 


HARDNESS  ASSURANCE  LEVELS 
IONIZATION  EFFECTS**  NEUTRONS 

NO  RHA  NO  RHA 

3  X  10*  RAD(SI)  2  X  10^* 

1  X  10*  RAD(SI)  2  X  10^2 

1  X  10*  RAD(SI)  2  X  10^2 

1  X  10*  RAD(SI)  2  X  IQi* 


*** 


*  FOR  MICROCIRCUITS ,  THE  LETTER  DESIGNATOR  FOR  RHA  PARTS  REPLACES  THE 
SLASH  IN  THE  PART  NUMBER;  E.G.  38510B291A  IN  PLACE  OF  38510/291A. 
FOR  DISCRETE  DEVICES  IT  IS  A  SUFFIX,  E.G.  JANTXVE. 

**  AT  A  DOSE  RATE  OF  200  RADS(SI)/SEC  PLUS  OR  MINUS  100  RADS(SI)/SEC. 
***  1  MEV  SILICON  DISPLACEMENT  DAMAGE  EQUIVALENT  NEUTRON  FLUENCE. 


^.3  Lot  acceptance  tests.  Lot  acceptance  tests  (LAT)  are  used  In  the 
military  standard  (MIL  STD)  system  for  determining  whether  a  production  lot  is 
of  high  enough  qtiallty  so  that  it  can  be  shipped  by  the  vendor.  The  tests  are 
of  a  statistical  nature  and  fall  Into  two  general  categories:  a)  attribute  tests 
and  b)  variables  tests. 

^•3.1  Attribute  tests  are  the  tests  most  often  used  In  the  MIL  STD 
procurement  system.  For  this  reason,  these  guidelines  will  be  directed  at  the 
use  attribute  tests  for  lot  acceptance.  Variables  tests,  on  the  other  hand,  are 
often  used  for  qualifying  and  accepting  system  production  parts  and,  for  such 
use ,  may  depend  on  the  particular  hardness  assurance  and  derating  procedures  that 
the  given  system  Is  using.  Variables  tests,  therefore,  more  properly  should  be 
discussed  as  part  of  a  system  hardness  assurance  guideline  document  and,  for  that 
reason,  are  not  discussed  In  detail  here. 

4. 3. 1.1  Advantages  and  disadvantages  of  attribute  tests .  An  advantage  of 
the  attribute  tests  is  that  the  data  collection  and  analysis  are  simple  and 
relatively  inexpensive.  A  further  Important  advantage  Is  that  the  test  is 
distribution  free  (i.e.  no  assumptions  are  required  about  probability 
distributions).  However,  a  major  drawback  Is  that  the  tests  require  inordinately 
large  sample  sizes  if  significant  survival  probability  requirements  are  imposed. 
For  example,  to  check  with  90%  confidence  that  a  lot  has  at  least  99%  survivable 
parts,  I.e  that  the  LTPD  Is  1%  or  less.  It  Is  necessary  to  test  a  sample  of  at 
least  231  parts  with  no  failures.  It  Is  Interesting  to  note  that  attribute  tests 
can  be  applied  to  attributes  that  can  assume  more  than  Just  the  two  values  of 
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pass  or  fall.  Acceptance  criteria  for  lots  can  also  be  based  on  a  mix  of 
attributes  that  may  be  desired  (e.g.  with  confidence  C  the  probability  of  a  given 
attribute  must  be  between  49.9%  and  50.1%). 

4. 3. 1.2  The  n/c  test.  The  most  common  type  of  attribute  test  is  the  so 
called  n/c  test  where  a  sample  of  n  parts  is  tested  and  if  more  than  c  of  the 
parts  fail  the  test  then  the  lot  is  rejected.  The  significant  test  result  for 
each  part  is  the  attribute  of  passing  or  failing  and  hence  the  term  attribute 
test.  The  usual  failure  criteria  for  parts  are:  a)  functional  failure  or  b) 
deterioration  of  any  critical  parameter  past  an  acceptable  limit. 

4. 3. 1.3  Multiple  sampling.  Some  attribute  tests  allow  the  drawing  of  extra 
samples  if  lots  fail  on  the  first  try  (multiple  sampling  plans) .  A  common 
multiple  sampling  plan  used  in  the  MIL  STD  system,  for  example,  is  a  test  of  11 
parts  where  the  lot  is  passed  if  there  are  no  failures.  Then,  if  there  is 
exactly  one  failure,  an  additional  7  parts  may  be  sampled  and  tested  with  no 
further  failures  allowed.  If  there  is  a  total  of  two  or  more  failures,  the 
original  lot  is  rejected. 

4.3.2  Confidence  and  probability.  The  results  of  the  MIL  STD  attribute  lot 
acceptance  test  are  phrased  in  terms  of  a  confidence  C  which  is  the  chance  of 
rejecting  a  lot  where  there  is  less  than  a  probability  P  that  any  part  in  the  lot 
can  pass  the  test  conditions. 

4. 3. 2.1  Usual  confidence  and  probability.  In  the  MIL  STD  system,  the 
significance  of  an  n/c  attribute  test  is  usually  given  in  terms  of  90  percent 
confidence  and  a  quantity  called  the  lot  tolerance  percent  defective  (LTPD)  which 
may  be  found,  for  example,  in  Table  B-1  in  MIL-M-38510 ,  the  general  specification 
for  microcircuits.  These  tests  are,  therefore,  commonly  also  referred  to  as  LTPD 
tests.  Thus,  for  example,  the  LTPD  associated  with  an  11/0  LTPD  test  may  be 
found  in  the  table  to  be  20%.  An  11/0  LTPD  test  will  therefore  provide  90 
percent  confidence  that  a  lot  with  20%  defective  parts  or  greater  will  be 
rejected.  In  terms  of  the  probability  P  mentioned  in  the  preceding  paragraph, 
LTPD  eqxials  1  minus  P. 

4.3.3  Variables  tests .  Variables  LAT  tests  measure  a  post-irradiation 
parameter  and  apply  statistical  analysis  using  approximate  or  exact  probability 
distributions  to  describe  the  variability  of  that  partuneter  (hence  the  term 
variables).  Usually  pre-irradiation  values  are  also  measured  and  they  often 
figure  in  the  analysis  (e.g.  when  the  quantity  of  interest  is  the  change  in  the 
parameter,  sometimes  referred  to  also  as  the  parameter  "delta").  Sometimes  the 
variable  of  concern  is  the  stress  to  failure. 

4. 3. 3.1  Advantages  and  disadvantages  of  variables  tests.  The  major 
advantage  of  variables  tests  is  that  a  high  probability  requirement  may  be 
Imposed  even  if  the  sample  size  is  only  moderate  (typically  10-50  parts).  A 
disadvantage  is  that  the  measurements  and  analysis  are  generally  more  complex  and 
expensive.  The  major  disadvantage,  however,  is  that  assumptions  must  be  made 
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about  the  probability  distribution  governing  the  failure  of  parts.  If  the 
distribution  is  very  well  known,  then  this  technique  can  yield  high  probabilities 
with  high  confidences.  However,  it  is  usually  difficult  to  know  the  "wings"  of 
the  distribution  accurately.  Therefore,  extrapolations  to  excessively  high 
probabilities  (such  as  0.999}  based  on  experience  with  only  a  few  parts  must  be 
regarded  with  skepticism.  It  is  important  to  note  that,  for  some  kinds  of 
electronics  effects  (e.g.  latchup) ,  there  is  very  little  reliable  Information 
about  the  nature  of  the  governing  probability  distribution. 

^•3.3.2  Variables  tests  depend  on  tolerance  limits  which  must  be  compared 
with  the  given  specifications  for  the  parameter  or  stress.  A  lot  is  rejected  if 
the  parameter  in  question  deteriorates  beyond  an  acceptable  limit  (or  if  the 
stress  to  failure  is  below  the  required  specification) .  The  tolerance  limits  are 
chosen  such  that  with  confidence  C,  the  probability  is  at  least  P  that  parts  will 
not  fall  the  test.  The  exact  phrasing  of  the  outcome  of  a  variables  test  is  very 
similar  to  that  for  the  attribute  test,  namely: 

4. 3. 3. 3  Confidence and  probability  for  variables  tests.  There  is  a 
confidence  C  of  rejecting  a  lot  if  there  is  less  than  probability  P  that  any  part 
in  the  lot  will  meet  the  specified  parameter  (or  stress)  tolerance  limit. 

4. 3. 3. 4  Lognormal  distribution.  The  most  commonly  assumed  distribution  for 
the  testing  of  electronic  parts  is  the  lognormal  distribution  (in  the  lognormal 
distribution  it  is  the  logarithm  of  the  quantity  that  is  normally  distributed) . 
The  normal  distribution  is  also  frequently  used.  Other  probability  distributions 
which  have  been  suggested  for  some  circumstances  are  the  Weibull  distribution  and 
the  extreme  value  distribution.  If  enough  parts  have  been  tested,  the 
characterization  data  can  be  examined,  perhaps  with  a  chi-square  test,  to  see 
which  type  of  distribution  fits  best. 

4. 3.3.5  Ramifications  to  variables  tests.  As  in  the  case  of  attribute 
tests ,  there  are  many  ramifications  to  the  use  of  variables  tests .  In  some  cases 
(e.g.  step— stress  tests)  the  uncertainty  in  the  measured  parameter  or  stress  can 
be  of  importance.  In  other  cases  a  two-sided  tolerance  limit  may  be  of 
importance.  Because  MIL  STD  device  specifications  use  attribute  (LTPD)  tests 
almost  exclusively,  this  document  will  not  address  the  ways  in  which  variables 
tests  may  be  used.  It  should  be  recognized,  however,  that,  when  systems  require 
very  hi^  plecepart  survival  probabilities,  variables  tests  for  lot  acceptance 
may  become  necessary. 

4.3.4  Sampling  statistics.  The  word  confidence,  as  used  here,  refers 
strictly  to  sampling  statistics  and  is  the  chance  of  rejecting  a  lot  where  there 
is  less  than  a  probability  P  that  any  part  in  the  lot  can  pass  the  test 
conditions.  The  general  practice  is  to  take  this  confidence  as  the  confidence 
that  shipments  will  be  acceptable.  This  latter  use  of  the  word  confidence  is  a 
matter  that  is  being  discussed  in  the  still  somewhat  controversial  subject  of 
Bayesian  statistics  and  is  beyond  the  scope  of  this  docviment.  The  important 
point  is  that  the  latter  confidence  may  only  be  approximated  and  involves 
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judgmental  decisions.  A  common  practice  in  hardness  assurance  is  to  take  the 
sampling  statistics  confidence  as  the  confidence  that  accepted  parts  will 
survive.  This  is  an  approximation  which,  though  usually  valid,  may  sometimes 
lead  to  error.  Perhaps  the  best  Justification  for  such  an  approximation  is  that 
it  is  ustially  not  the  major  factor  which  limits  the  accuracy  and  confidence 
levels  associated  with  system  svirvivability  estimates  and  risk  assessments. 
Often,  a  more  difficult  problem,  for  example,  is  simulation  fidelity  (does  the 
test  accurately  represent  the  threat  environment  that  is  specified  for  the 
system) . 

^'3.5  Reasonablllitv  of  risk  assessment  results.  The  major  point  of  the 
above  discussions  is  to  show  the  approximate  nature  of  risk  assessment. 
Excessively  high  survival  probabilities,  such  as,  for  example,  a  survival 
probability  of  0.999999  (so  called  6-nines)  may  be  unreliable  for  a  number  of 
reasons.  First  of  all,  such  numbers  can  only  be  obtained  from  extrapolations 
based  on  an  exact  knowledge  of  the  probability  distribution.  However,  because 
radiation  tests  use  modest  numbers  of  parts  and  the  parts  which  are  tested  cannot 
be  used  afterwards,  knowledge  of  the  assumed  probability  distribution  is  never 
adequate  for  such  extrapolations.  Thus,  for  exaotple,  some  accidental  occurrence 
during  the  production  of  the  parts  may,  with  a  probability  exceeding  1  minus 
0.999999,  prodxice  a  part  which  does  not  fit  the  probability  distribution  assumed 
for  the  majority  of  the  parts;  the  presence  of  such  parts  cannot  be  detected  with 
confidence  by  the  sample  sizes  which  are  typical  of  radiation  tests.  At  the  6- 
nlnes  level  also,  human  errors  in  making  measurements  or  in  handling  the  parts 
can  become  a  significant  failure  mode  which  is  totally  outside  the  assumed  causes 
of  part  failure. 

Part  Characterization.  FIGURE  2  shows  the  major  steps  by  which 
characterization  data  is  measured  and  used  for  calculating  the  RHA  end-point 
limits  (RHEPL)  which  are  then  used  in  the  device  procurement  specification  for 
radiation  qualification  and  lot  acceptance  tests  (LAT) .  An  important  point  shown 
in  FIGURE  2  is  that  an  objective  must  be  selected  for  the  LAT  (or  QCI)  tests  and 
that  the  RHA  end-point  limits  (RHEPL)  are  then  calculated  to  meet  that  objective 
(the  acronym  PIPL,  standing  for  post-irradiation  parameter  limits,  is  also  used) . 
Thus,  for  example,  the  objective  could  be  to  set  the  end-point  limits  so  that, 
with  90  percent  confidence,  at  least  90  percent  of  future  lots  may  be  expected 
to  pass  a  22/0  test.  Pre-rad  measurements  are  essential  when  the  change  (delta) 
in  the  value  of  a  parameter  is  of  greater  Interest  than  the  absolute  value  of  the 
parameter.  Although  they  may  not  be  essential  when  the  absolute  value  of  the 
parameter  is  the  important  quantity,  they  are  nevertheless  highly  desirable. 
FIGURE  2  and  subsequent  discussions,  therefore,  include  pre— rad  measurements  as 
part  of  the  characterization  procedure. 
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AND  F08T-  RADIATION  DATA 


SELECT  DESIRED  LOT  ACCEFTANCE  FRORABILITY 
AND  CONFIDENCE  LEUELS  FOR  LTFD  TESTS 


SELECT  RHA  LTFD  TEST  REQUIREnEKTS 


CALCULATE  RHEFL  FROH  CHARACTERIZATION  DATA 


INCORFORATE  THE  RHEFL  VALUES  AND  LTFD  TEST  RE- 
QUIREI1ENTS  INTO  THE  DETAILED  SPECIFICATIONS 


COORDINATE  DETAILED  RHA  SPECIFICATION 


INCORPORATE  RHEFL  AND  PUBLISH  THE  DETAILED  RHA  SPECIFICATION 


FIGURE  2.  Flow  diagram  for  characterizing  and  specifying  the  performance  of 
semiconductor  devices  In  a  radiation  environment. 
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4.5  Considerations  bevond  the  scope  of  this  document. 


4.5.1  Incorporating  additional  radiation  and  non-radiation  information. 
If  information  is  obtained  in  addition  to  that  provided  by  the  given  LTPD  test 
against  the  RHEPL  contained  in  the  detailed  device  specification,  then,  for 
particular  system  applications,  there  are  several  methods  by  which  RHA  part 
survival  probabilities  can  be  obtained  which  are  higher  than  those  which 
correspond  to  the  LTPD  test  alone.  These  methods  include:  the  use  of  the  next 
higher  radiation  level  RHA  part,  radiation  overtests,  the  use  of  lot  rejection 
information,  the  use  of  variables  testing  for  LAT,  and  the  use  of  derated 
parameter  values.  As  an  example,  most  systems  manufacturing  companies  are 
already  familiar  with  use  of  derating  factors  for  temperature  and  aging. 
Radiation  derating  factors  may  be  used  in  much  the  same  way  and  can  thus  be 
fitted  into  existing  practices.  These  methods  properly  belong  in  a  document  on 
system  level  hardness  assurance  and  are  not  further  addressed  here. 

4.5.2  Further  complex  Questions.  The  statistical  questions  involved  in 
hardness  assurance  and  lot  acceptance  tests  range  in  complexity  from  cases  for 
which  the  analysis  methods  are  relatively  straightforward  to  those  for  which  the 
required  formalism  is  still  being  developed.  In  the  former  instances,  these 
guidelines  provide  simple  step  by  step  procedures  that  may  be  used  for  the 
analysis  of  the  characterization  data.  In  the  latter  Instances,  which  are  beyond 
the  scope  of  this  document,  these  guidelines  can  only  suggest  approaches  which 
are  overly  conservative  or  recommend  chat  statisticians  be  consulted  for  less 
conservative  but  still  valid  analyses.  In  the  latter  category  are  cases  where 
the  within-lot  variations  are  comparable  to  lot-to-lot  variations  (in  this  case 
the  definition  of  sample  size  can  become  uncertain) ,  cases  of  combined 
environments  or  combined  parameters,  and  cases  where  not  enough  data  is 
available.  The  1987  paper  by  Namenson  and  Arlmura,  listed  in  Section  2.2,  may 
be  consulted  for  an  approach  to  multilot  and  multiparameter  data  analysis. 
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5  DETAILED  REQUIREMENTS 


5.1  Characterization  of  pieceoart  performance  in  radiation  environments. 
The  first  step  required  for  developing  a  device  specification  is  to  characterize 
the  radiation  response  of  the  part  t3rpe  in  question.  Such  a  characterization  for 
one  or  more  radiation  environments  requires  measurement  of  its  post-irradiation 
performance  and,  usually,  its  pre-irradiation  performance  as  well.  The  parts, 
generally,  should  be  characterized  through  to  failure  so  that  the  specifications 
can  be  written,  if  necessary,  for  the  maximum  capability  of  the  part  type.  For 
the  present  discussion  it  is  assumed  that  the  purpose  of  the  characterization 
measurements  is  to  support  the  calculation  of  parameter  end-point  limits  for 
qualification  and  LAT  tests  in  a  MIL  STD  detailed  specification  and  that 
satisfactory  data  for  the  part  type  in  question  does  not  already  exist.  In  this 
case,  the  characterization  measurements  should  be  such  as  to  permit  these 
calculations  to  be  made  for  the  radiation  levels  given  in  TABLE  1  (Section  4.2). 
Alternatively,  the  characterization  measurements  may  be  required  in  direct 
support  of  a  hardened  system  design  or  production  (custom  specifications) .  In 
this  latter  case,  the  parameters  which  are  measured  and  the  types  and  fluence 
levels  of  the  radiations  which  are  used  should  reflect  the  system  requirements 
and  the  specific  vendors  who  are  expected  to  be  suppliers  for  the  system.  The 
characterization  measurements  should  also  determine  the  radiation  levels  required 
to  caxise  part  failure;  such  information  can  be  important  for  estimating  the 
design  margin  when  the  part  is  used  in  some  particular  application. 

To  simplify  the  present  discussion^J^assum^tha^uI^averag^post— irradiation 
performances  of  a  part  type  may  differ  for  different  manufacturers  and,  for  a 
given  manufacturer,  may  differ  from  one  lot  to  another,  but  that  the  average 
radiation  response  characteristics  of  a  part  type,  for  a  given  manufacturer,  do 
not  significantly  change  with  time.  This  latter  assumption  may  not  always  be 
warranted.  It  is  needed  to  keep  the  initial  disctission  of  data  measurement  and 
analysis  free  of  the  additional  complexities  that  time  variations  produce. 


5.1.1  Documentation  of  characterization  information.  Experience  has  shown 
chat  the  analysis  of  radiation  data  at  a  future  time  or  by  persons  ocher  than 
chose  who  performed  the  measurements  is  enormovisly  facilitated  by  good 
documentation  of  the  test  results .  At  the  same  time ,  a  need  to  analyze 
previously  acquired  data  or  data  acquired  by  others  is  common.  For  these  reasons 
and  because  new  measurements  are  costly,  these  guidelines  recommend  Chat  Che 
overall  characterization  measurements  be  summarized  in  a  Characterization  Report 
and  chat  the  conditions  under  which  the  measurements  were  made  be  documented  in 
a  Test  Plan  which  is  included  in  Che  characterization  report.  These  documents 
should  furthermore  be  sufficiently  complete  so  that  independent  data  analyses  can 
be  performed  on  the  results  contained  therein. 

5. 1.1.1  Characterization  Report.  The  steps  required  to  characterize  the 
performance  of  a  part  are  shown  in  FIGURE  3  and  discussed  in  Che  sections  that 
follow.  In  general,  the  discussions  identify  the  decisions  that  have  to  be  made 
and  the  factors  that  should  be  considered  for  each  topic.  The  information 
developed  in  these  steps  is  Chen  to  be  documented  in  the  characterization  report. 
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Some  of  these  Items  will  also  be  listed  In  the  overall  test  plan  for  the 
measurements.  This  approach  preserves  all  the  information  that  is  developed  in 
the  program  but  keeps  the  test  plan  as  short  as  possible.  A  check-off  list  of 
all  the  steps  required  for  the  characterization  measurements  is  given  in  TABLE  2. 


FIGURE  3.  Flow  diagram  for  characterizing  the  performance  of 


semiconductor  devices  in  a  radiation  environment. 
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TABLE  2.  Step  by  step  procedure  for  characterization  measurements. 


Check-Off  Item 

1.  Start  preparing  the  characterization  report.  The  characterization 
report  will  document  the  information  issuing  from  steps  2  through  25 . 

2.  Identify  the  manufacturer. 

3.  Specify  a  way  of  selecting  a  random  sample. 

4.  Specify  sample  size  and  distribution  in  time. 

5.  Select  the  samples;  include  extra  parts  for  equipment  checkout. 

6.  Record  device  identity  for  each  sample  device;  include  traceability 
to  process  lot  if  applicable. 

7.  Select  radiation  environments. 

8.  For  each  radiation  environment,  select  test  parameters. 

9.  For  each  parameter,  select  test  conditions;  if  possible,  post  rad 
conditions  should  be  the  same  as  the  pre  rad  conditions.  Include 
location  in  test  cell,  bias  conditions,  etc. 

10.  Specify  a  data  format. 

11.  Start  preparing  the  test  plan.  The  test  plan  will  be  a  subsection 
of  the  characterization  report  that  will  document  the  information 
issuing  from  steps  12  through  24. 

12.  Perform  pre-rad  measurements 

13.  Examine  measurement  results  for  bad  devices  or  data  outliers. 

14.  Replace  bad  device  or  repeat  measurement,  whichever  is  indicated. 

15.  Record  measurement  conditions  and  results  according  to  the  specified 
data  format. 

16.  For  each  radiation  environment,  select  irradiation  facility. 

17.  For  each  radiation  environment,  specify  irradiation  conditions,  and 
device  operating  conditions  during  the  irradiation. 

18.  Estimate  device  failure  levels  including  possible  abrupt  failures. 

19.  Specify  irradiation  step-stress  fluences  or  dose  rates  to  ensure  an 
adequate  set  of  measurements. 

20.  As  required,  check  out  the  irradiation  procedure,  test  fixture,  and 
measuring  equipment  with  spare  parts. 

21.  Perform  radiation  exposures  and  record  irradiation  conditions. 

22.  As  required,  perform  in  flux  or  post  radiation  measurements. 

23.  Examine  meastirement  results  for  bad  devices  or  data  outliers. 
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5. 1.2  Selection  of  manufacturers.  For  a  selected  part  type,  the  potential 
manufacturers  to  supply  that  part  for  government  procurement  should  be 
identified.  If  the  part  type  is  already  a  JAN  type,  the  QPL  will  serve  as  a 
starting  point.  DESC  and  the  preparing  activities  for  the  military  standard 
procurement  system  (RADC,  SPAUAR,  and  NASA)  should  be  consulted,  however,  to 
learn  whether  additional  vendors  might  be  in  the  process  of  qualifying  for  that 
psrt  type.  If  the  part  in  question  is  to  be  supplied  to  a  standardized  military 
drawing  (SMD)  or  other  government  specification,  then  the  list  of  vendors  will 
need  to  be  obtained  either  from  DESC  or  the  cognizant  project  office. 

5.1.3  Selection  of  random  test  samples.  Sample  parts  for  testing  should 
be  selected  at  random.  Becatise  of  cost,  part  availability,  schedules,  etc.,  it 
may  not  always  be  practical  to  obtain  an  adequate  set  of  samples .  The  procedures 
discussed  under  data  analysis  in  Section  5.2.4,  will  still  apply  but  will  produce 
increased  uncertainties  in  the  results  which  will  reflect  the  inadequacy  of  the 
sample  of  parts  which  was  used  for  the  characterization.  One  way  of  saving  costs 
might  be  the  use  of  parts  which  have  passed  Group  A  electrical  performance  tests 
and  alternate  visual  criteria;  this  possibility  is  mentioned  for  VLSI  parts  in 
particular  but  could  have  application  whenever  the  cost  of  sample  parts  is  a 
limiting  factor.  It  is  always  advisable  that  the  test  parts  have  gone  through 
bum- in. 


5.1.4  Obtaining  samples.  Once  the  vendors  have  been  identified,  samples 
of  parts  should  be  obtained  from  each  vendor.  For  each  vendor,  the  samples  need 
to  be  taken  in  random  fashion  from  each  lot  of  parts  that  is  being  used  and  the 
lots  need  to  be  selected  so  that  they  represent,  as  accurately  as  possible, 
production  characteristics .  The  production  variations  which  are  of  concern  are : 

a)  variations  over  time,  b)  variations  from  one  wafer  lot  to  another,  c) 
variations  from  one  wafer  to  another  in  a  wafer  lot,  and  d)  variations  of  the 
devices  from  one  region  of  the  wafer  to  another. 

A  few  extra  samples  should  be  obtained  to  cover  the  possibility  that  some 
devices  may,  in  later  measurements,  prove  to  be  either  bad  devices  or  outliers 
which  will  need  to  be  replaced. 

5. 1.4.1  Recommended  sample  size.  To  meet  the  concerns  listed  in  the 
previous  section,  the  following  quantities,  for  each  vendor  and  for  each 
radiation  environment,  are  recommended; 

a)  Three  wafer  lots  taken  at  least  one  month  apart. 

b)  Five  wafers  per  wafer  lot. 

c)  Five  devices  per  wafer  taken  one  from  each  qxiadrant  and  one  from 
the  center. 

A  few  extra  samples  should  be  obtained  to  cover  the  possibility  that  some 
devices  may,  in  later  measurements,  prove  to  be  either  bad  devices  or  outliers 
which  will  need  to  be  replaced.  See  Section  5. 2. 4. 1.1  for  a  discussion  of  bad 
devices  or  outliers. 
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5. 1.4. 2  Reduced  sample  sizes.  It  should  be  understood  that,  in  cases  where 
the  reconunended  quantities  prove  to  be  impractical  because  of  cost  or  schedule 
impacts,  the  vendor  may  propose  reduced  sampling  requirements.  Reduced 
requirements  may  be  acceptable,  for  example,  where  data  exists  showing  the 
radiation  response  of  the  part  type  in  question  to  be  stable  and  predictable  over 
long  periods  of  time  or  that  the  part  response  will  be  similar  to  that  of  another 
part. 


5. 1.4. 3  Minimum  sample  sizes.  The  sample  size  recommended  in  the  previous 
section,  for  each  vendor  and  radiation  environment,  is  a  total  of  75  devices. 
If  that  number  of  devices  cannot  be  made  available,  then  reduced  sample  sizes 
will  have  to  be  considered.  Roughly  speaking,  25  is  the  minimum  sample  size 
which  should  be  used  for  a  characterization  measurement  —  the  more  parts  the 
better.  For  this  minimum  sample  size,  the  recommendation  is  that  5  different 
lots  be  sampled  and  that  5  samples  per  lot  be  taken.  A  sample  of  25  will  at 
least  allow  a  goodness  of  fit  test  to  determine  whether  the  part's  behavior 
belongs  to  a  well  defined  statistical  distribution.  If  the  relevant  parameters 
degrade  gracefully  with  radiation  and  if  the  probability  distribution  is  known 
well  enough,  then  5  or  even  as  few  as  3  parts  from  each  lot  might  be  adeqviate. 
Larger  sample  sizes  will  reduce  vincertainties  about  the  nature  of  the  governing 
probability  distribution  and  will  thereby  permit  better  performance  estimates  for 
the  parts  remaining  in  the  population  from  which  the  sample  was  drawn.  Sample 
sizes  which  are  too  small  may  result  in  characterization  data  with  large 
statistical  uncertainties.  These  uncertainties  can  result  in  overly  conservative 
decisions  about  how  the  part  should  be  used  or  purchased  and  may  lead  to  costs 
which  will  more  than  undo  the  savings  produced  by  the  small  sample  size.  No 
simple  prescription  for  selecting  sample  size  can  be  given.  Relevant  factors, 
in  addition  to  part  cost,  which  will  affect  how  large  the  sample  size  should  be 
(for  each  vendor),  will  include  the  amount  of  previous  knowledge  available  about 
the  part  and  the  variability  of  the  part's  performance  on  a  single  wafer,  within 
a  wafer  lot,  and  from  lot  to  lot.  If  lot  to  lot  variability  is  large  compared 
to  variability  within  one  lot  (a  not  uncommon  occurrence)  then,  in  the  data 
analyses  that  are  performed,  the  sample  size  effectively  becomes  the  number  of 
lots  and  not  the  number  of  parts  tested. 

5. 1.4. 4  Sample  sizes  in  characterization  report.  The  number  of  samples  to 
be  tested,  for  each  vendor,  Is  to  be  Included  in  the  characterization  report  and 
in  the  test  plan. 

5.1.5  Device  Identity  number  and  pedigree.  Serious  problems  in  analyzing 
data  have  also  been  experienced  because  the  device  that  was  tested  was  not 
adequately  Identified.  The  information  needed  to  establish  an  adequate  device 
identity  is  shown  in  TABLE  3.  This  information  should  be  recorded  in  the 
characterization  report.  As  disctissed  in  Section  5. 1.7. 5  on  data  format,  a 
subset  of  the  TABLE  3  information  should  be  included  when  the  actvial  test  data 
is  recorded  and  tabulated  in  the  characterization  report.  It  should  be 
recognized  that  radiation  response  characteristics,  especially  for  ionizing 
radiation  dose,  can  be  very  sensitive  to  processing  conditions.  Thus,  to  the 
maximum  extent  practicable,  all  processing  steps  and  equipment  should  be 
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TABLE  3.  Device  identity  information. 


a.  Device  identification  number 

b.  Generic  part  number 

c.  JAN  part  number  (if  applicable) 

d.  Name  of  the  manufacturer 

e.  Lot  date  code 

f.  Lot  number  (wafer  lot  number,  if  available) 

g.  Wafer  number  (if  applicable) 

h.  Serial  nuiiA>er  (for  each  device) 

i .  Package  type 

j.  Special  selection  criteria  (if  applicable) 


documented  for  the  lot  under  test.  Whenever  possible  also,  the  lot  under  test 
should  be  a  wafer  lot,  because  a  wafer  lot  is  defined  as  one  for  which  all 
processing  steps  are  the  same  and  performed  with  the  same  equipment.  In  the  case 
of  JAN  Class  S  devices,  wafer  identity  is  also  available  and  should  be  recorded. 

5.1.6  S-election  of  radiation  environments .  The  characterization  report  and 
the  test  plan  should  identify  the  radiation  environments,  such  as  neutrons; 
ionizing  radiation  dose  (total  dose)  and  dose  rate,  also  the  type  of  radiation 
,  i.e.  whether  electrons,  gamma  rays,  protons,  etc.,  and  heavy  ions  or  protons 
(single  event  upset  (SEU)),  the  radiation  facilities,  and  the  levels  that  will 
need  to  be  vised  for  the  testing.  This  step  should  precede  that  of  selecting  the 
test  parameters  to  ensure  that  no  parameters  are  overlooked  which  might  be 
especially  sensitive  to  a  particular  radiation  environment.  For  the  two 
radiation  environments  of  principal  interest  here,  namely  neutrons  and  ionizing 
radiation  dose,  the  effects  are  expected  to  be  independent  and  characterization 
measurements  can  be  made  separately  for  the  two  environments.  If  the  effects 
from  two  environments  are  expected  to  depend  on  each  other,  as  for  example, 
temperature  effects  on  dose  rate  induced  latchup,  or  ionizing  radiation  dose 
effects  on  SEU  sensitivity,  then  each  part  will  have  to  be  exposed  to  both 
environments.  If  the  given  part  type  is  known  to  be  very  hard  to  a  particular 
radiation  environment  as ,  for  example ,  an  HOS  device  may  be  to  neutrons ,  then 
characterization  measurements  for  that  type  of  radiation  may  be  omitted.  Careful 
consideration  should  be  given,  however,  to  the  possibility  that  intentional  or 
parasitic  sensitive  elements  (e.g.  bipolar  devices  in  the  HOS  example  cited)  may 
be  present  in  the  device. 
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5.1.7  Selection  of  test  parameters.  All  the  parameters  that  are  to  be 
measured  need  to  be  selected  and  listed  in  the  test  plan.  Normally,  for  JAN 
parts,  the  subset  of  Group  A  electrical  parameters  that  are  known  to  be  sensitive 
to  the  selected  radiations  will  be  used.  In  some  instances,  the  change  in  the 
value  of  a  parameter  is  more  significant  than  the  absolute  value  of  the 
parameter.  For  those  cases,  the  test  plan  should  specify  that  the  change,  or 
delta,  in  the  parameter  value  be  measured. 

5. 1.7.1  Test  conditions  and  parameters  to  be  recorded.  The  parameters  or 
circuit  ftmctions  that  need  to  be  measured  will  have  to  be  identified  and  listed 
in  the  report  and  in  the  test  plan  along  with  the  experimental  conditions  under 
which  the  measurements  are  to  be  made.  The  specified  conditions  should  include 
both  the  nominal  and  the  worst  case  conditions  under  which  each  parameter  should 
be  measured.  They  should  also  include  such  quantities  as,  bias  voltages, 
operating  frequencies,  and  ambient  temperatures  before,  during,  and  after 
Irradiation.  For  the  worst  case  measurements,  the  report  should  contain 
recommendations  on  how  such  conditions  may  be  derived  or  at  least  bracketed 
(circuit  analyses  may  be  required  to  arrive  at  worst  case  conditions).  This 
section  of  the  report  may  also  be  used  to  identify,  for  each  parameter  that  is 
to  be  measured,  what  value  of  the  parameter  constitutes  device  failure.  Such 
definitions  are  needed  in  connection  with  the  fluence  to  failure  measurements 
recommended  earlier  for  permitting  estimates  of  design  margin  to  be  made  for 
Particular  applications.  Failure  may  be  either  parametric  failure,  loss  of 
functionality,  or  degradation  to  the  point  where  the  device  can  no  longer  be 
used. 


5. 1.7. 2  Sl?.l^.gtlvM  of  the  test  test  conditions  and  parameters.  The 
selection  of  test  conditions  and  test  parameters  will  depend  on  the  objectives 
of  the  test,  time  and  budget  constraints,  and,  possibly,  other  factors.  For  each 
test,  however,  the  information  listed  in  TABLE  4  should  be  recorded  in  the 
characterization  report.  As  is  discussed  in  Section  5. 1.7.5  on  data  format,  a 
subset  of  the  TABLE  4  information  should  be  included  when  the  actual  data  is 
tabulated  and  recorded  in  the  characterization  report.  Experience  gained  by  the 
DNA  sponsored  Electronic  Radiation  Response  Information  Center  (ERRIC)  is  the 
basis  for  most  of  the  recommendations  contained  in  these  guidelines  regarding 
data  requirements  and  format. 
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TABLE  4.  Pre-  and  post-lrradlatlon  measurement  Information. 


a.  Name  and  organization  of  test  engineer 

b.  List  of  equipment  used  for  measurements  and  calibration  procedures 

c.  Description  of  test  procedures  or  test  standards  used 

d.  Electrical  test  date 

e  Test  number 

f.  List  or  table  of  electrical  test  parameters  and  corresponding  bias 
conditions ;  (note  that  the  pre-rad  bias  conditions  should  be  used  for 
the  post-rad  measurements) 

g.  A  complete  time  history  of  the  individual  irradiations  and  the  post¬ 
irradiation  measurements  (especially  necessary  for  ionizing  radiation 
dose  (total  dose)  because  of  time  dependent  effects) 

h.  Ambient  and/or  case  temperature 

1.  Sample  size 

j .  Electrical  measurement  results  for  each  parameter 

k .  Narrative  Information 


5. 1.7. 3  The  narrative  information  listed  in  the  TABLE  4  is  used  to  provide 
a  further  description  of  the  device  and  the  test  conditions  required  for  each 
test. 


5. 1.7. 4  A  partial  list  of  MIL  STD  symbols  for  particular  device  parameters 
and  the  corresponding  symbols  used  by  ERRIC  is  given  in  APPENDIX  I.  A  standard 
list  which  all  users  have  agreed  to  use  does  not  exist  for  device  parameters. 
As  a  result,  different  symbols  are  sometimes  used  by  different  organizations  for 
the  same  device  parameter.  ERRIC  can  receive  data  labelled  in  any  way  and  uses 
a  narrative  coomient  as  part  of  the  data  storage  to  make  sure  the  device  parameter 
is  properly  identified.  If  data  processing  programs  are  used  for  analyzing  the 
behavior  of  particular  parameters,  provision  should  be  made  for  recognizing  some 
limited  set  of  the  syndiols  for  the  parameter  in  question  in  addition  to  those 
given  in  APPENDIX  I._ 

5. 1.7. 5  Specification  of  a  data  format.  Although  data  format  would  appear 
to  be  a  simple  matter,  past  experience  has  shown  that  severe  difficulties  can  be 
encoxmtered,  particularly  with  computerized  data  analysis  programs ,  if  a 
standardized  data  format  is  not  used  to  record  the  data.  Because  the 
accumulation  of  radiation  response  information  in  a  centralized  data  bank  can 
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help  avoid  duplication  of  such  measurements  and  is  therefore  highly  desirable, 
the  data  format  recommended  here  is  the  one  used  by  ERRIC. 

5. 1.7. 5.1  Definition  of  a  test  number  in  ERRIC.  In  ERRIC,  a  xinique  test 
number  is  assigned  to  both  the  pre-  and  post— rad  measurements  that  are  made  on 
a  single  parameter  under  a  single  set  of  bias  or  operating  conditions,  for  all 
devices  in  the  sample,  and  for  all  the  radiation  fluences  used.  The  information 
which  is  common  to  a  unique  test  number  and  which  can  be  used  therefore  as  the 
heading  for  that  test  nxunber  is  listed  in  TABLE  5.  Pre-  and  post-rad  test 
results  obtained  for  all  the  sample  devices  that  are  covered  by  the  same  heading 
are  then  listed  under  a  single  test  niuaber. 

TABLE  5.  Example  of  erric  test  data. 


Test  number:  1  :  Test  date:  10-21-85 

Type  of  radiation:  neutrons :  Radiation  facility:  PMI 

Device  type:  DAC-08 :  Manufacturer:  ABC:  Lot  date  code:  8429 

Parameter:  IREF:  Units: 

Test  conditions  and  narrative  information: 

a.  bit  high  speed  multiplying  D/A  converter  (device  description  included 
only  for  test  number  1) 

b.  Reference  bias  current  (parameter  description) 

c.  V+  -  15  V,  V - 15  V 

d.  Measured  24  hours  after  the  end  of  the  irradiation. 

MEASUREMENT  DATA 

DEVICE  RADIATION  LEVEL 


SERIAL 

NUMBER 

PRE-RAD 

2.00E12 

6.00E12 

1.00E13 

0271 

2.00 

2.00 

2.00 

2.00 

0272 

2.00 

2.00 

2.00 

2.00 

•  •  •  • 

•  •  ■  • 

•  •  •  • 

»  •  •  • 

•  •  •  • 

TABLE  6  shows  an  example  of  how  ERRIC  data  is  recorded. 

For  test  number  2  and  subsequent  tests  made  on  the  same  devices  and  for  the 
same  irradiation  conditions,  the  first  three  lines  of  the  heading  need  not  be 
repeated.  Thus,  the  data  for  test  number  2  could  be  recorded  as  in  TABLE  7; 
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TABLE  6.  InformaClon  common  to  a  unique  test  number  in  erric. 


a.  Single  type  of  radiation  and  corresponding  test  facility 

b.  Single  part  type 

c.  Single  manufacturer 

d.  Single  test  date  (post-rad) 

e.  Single  lot  date  code 

f.  Single  wafer  lot  number  (if  available) 

g.  Single  parameter  measured  and  associated  bias  or  operating  conditions 

h.  Narrative  Information  (should  include  information  about  the 
irradiation  conditions  and  time  history  of  the  irradiations  and  the 
electrical  measurements) 


TABLE  7 .  Erric  data  format  for  test  number  2  in  a  series . 


Test  number:  2  :  Test  date:  10-21-85 

Repeating  test  information:  Same  as  lines  2  and  3  for  test  number  1 
Parameter:  IZS  :  Units: 

Test  conditions  and  narrative  information: 

a.  Zero  scale  current  (parameter  description) 

b.  V+  -  15V,  V-  —15  V. 

c.  Time  of  measurements  referred  to  the  end  of  the  irradiation. 

MEASUREMENT  DATA 

DEVICE  RADIATION  LEVEL 

SERIAL  PRE-4tAD  2.00E12  6.00E12  1.00E13 

NUMBER 
0271 
0272 


4.20E-5  2.00E-5  6.50E-5  1.67E-4 

-2.00E-6  4.70E-5  9.00E-5  2.54E-4 


5.1.8  Test  Plan.  FIGURE  3  (Section  5. 1.1.1)  and  TABLE  2  (Section  5. 1.1.1) 
show  the  major  steps  involved  in  characterizing  the  radiation  response  of  a  part. 
Also  shown  are  the  steps  which  are  included  in  the  test  plan.  The  test  plan 
should,  as  a  minimtus,  address  the  pre-irradiation  measurements  to  be  made,  the 
irradiations  to  be  performed,  the  post  irradiation  measurements  to  be  made,  and, 
finally,  should  specify  the  data  documentation  requirements  and  format. 

5. 1.8.1  Advantages  of  a  good  test  plan.  The  development  of  a  detailed  test 
plan  is  recommended  as  a  way  to  make  characterization  measurements  efficient. 
A  good  test  plan  will  minimize  costs  by  specifying  the  data  set  which  will  meet 
the  objectives  of  the  measurements  and  by  optimizing  the  use  of  radiation  test 
facilities  and  part  samples.  The  test  plan  should  also  facilitate  subsequent 
data  analysis  by  specifying  a  data  format  which  will  be  easy  to  use  and  will 
protect  against  inadvertent  omission  of  required  data  points  by  providing  a  check 
list  for  the  measurements. 

5. 1.8. 2  Pre-irradiation  measurements  and  data  examination.  If  the  pre- 
irradiation  characteristics  are  measured,  the  measurements  should  be  made 
according  to  the  conditions  listed  in  TABLE  4  (Section  5. 1.7. 2)  and  recorded  in 
the  format  given  in  TABLE  6  (Section  5. 1.7. 5.1)  and  TABLE  7  (Section  5. 1.7. 5.1). 
The  data  should  be  examined  to  make  sure  that  there  are  no  bad  measurements, 
devices  or  data  outliers  in  the  sample.  This  examination  is  Important  to  keep 
bad  data  from  being  entered  into  the  report.  If  bad  or  anomalous  data  are  taken, 
this  fact  together  with  an  explanation  of  why  the  data  is  not  being  used  should 
be  recorded  in  the  characterization  report  even  if  the  data  are  not  sent  to  ERRIC 
or  some  other  data  bank.  The  annual  books  of  ASTM  standards,  listed  in  Section 
2.2,  contain  many  of  the  test  methods  that  can  be  used  for  making  radiation 
response  measurements. 

5.1.8. 3  Caution  should  be  exercised  when  devices  are  handled,  particularly 
with  regard  to  pin  alignment  in  the  holding  fixture  and  when  the  devices  are 
attached  to  the  test  circuit.  Bias  voltages  should  be  off  during  attachment. 
ESD  handling  procedures  should  be  observed  for  the  class  of  parts  being  tested. 

5. 1.8. 4  Irradiation  conditions .  The  irradiation  conditions  and  the  reasons 
for  selecting  them  should  be  discussed  in  detail  in  the  characterization  report 
and  summarized  in  the  test  plan.  The  information  to  be  included  in  the  test  plan 
is  listed  in  TABLE  8.  The  various  questions  that  need  to  addressed  in  selecting 
the  irradiation  conditions  are  discussed  in  the  sections  which  follow.  For  each 
of  the  selected  radiation  environments,  the  plan  should  include  the  test  facility 
selected,  the  responsible  organization  and  personnel,  recommended  dosimetry 
techniques,  the  maximum  fluence  or  ionizing  radiation  dose  to  be  reached 
(whenever  possible,  the  radiation  levels  should  be  high  enou^  so  that  most  of 
the  test  devices  can  be  made  to  fail) ,  and  whether  device  measurements  will  be 
made  "in  flvix",  i.e.  while  the  device  is  being  irradiated,  or  whether  a  series 
of  discrete  irradiations  with  measurements  following  each  irradiation  will  be 
made.  In  the  latter  case,  which  is  sometimes  called  step  stress  testing,  the 
test  plan  should  specify  the  nximber  of  irradiations  to  be  made  and  the  ionizing 
radiation  dose  or  fluence  for  each  Irradiation  in  the  series.  Because  time 
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TABLE  8.  Irradiation  conditions. 


a.  Name  of  the  radiation  test  facility. 

b.  Name  and  organization  of  radiation  test  engineer 

c.  List  of  dosimeter  types  and  readout  equipment 

d.  Date  of  irradiations 

e.  Irradiation  run  number 

f.  Number  of  samples  in  each  irradiation 

g.  Placement  of  samples  with  respect  to  the  radiation  source 

h.  Radiation  facility  operational  mode 

i.  For  each  irradiation,  flux  or  dose  rate,  fluence  of  ionizing 
radiation  dose,  start  and  stop  times,  and  ambient  or  device  case 
temperature 

j  .  Device  bias  conditions  and  whether  In-fltix  measurements  will  be  made 

k.  Start  and  stop  times  for  post  irradiation  measurements,  including 
measurements  made  during  a  series  of  radiation  exposures 
(particularly  important  for  ionizing  radiation  dose  because  of  the 
time  dependent  effects  which  can  occur) 


dependent  effects  can  be  very  important,  especially  for  ionizing  radiation  dose, 
a  complete  time  history  of  the  Irradiations  and  the  post— radiation  or  in-flux 
measurements  should  be  recorded.  This  time  history  should  Include  the  start  and 
stop  times  of  both  the  Irradiations  and  the  measurements  as  well  as  the  rate  at 
which  the  ionizing  radiation  dose  was  delivered.  The  test  plan  should  state, 
vdienever  possible,  that  a  standard  irradiation  test  method  should  be  used  and 
give  the  appropriate  reference. 

5. 1.8. 5  Estimates  of  device  failure  levels.  To  reduce  costs  and  to 
optimize  the  usefulness  of  the  data  obtained,  it  is  Important  to  estimate  the 
expected  failure  levels  for  the  selected  device  type  and  radiation  environment 
before  full  scale  characterization  measurements  are  begun.  Device  failure  will 
usually  be  defined  as  parametric  failure  but  may  sometimes  be  defined  as 
functional  failure.  In  the  case  of  parametric  failure  it  is  well  known  that 
different  failure  limits  can  be  defined  for  different  device  applications.  For 
the  present  purpose,  engineering  judgement  should  be  used  to  define  a  failure 
limit  for  each  parameter  being  measured.  The  important  point  is  that  the  lowest 
stress  level  which  renders  the  device  unacceptable  should  be  considered  as  the 
failure  level.  Estimates  of  failure  levels  will  have  to  based  either  on  previous 
experience  and  data  or  on  measurements  on  a  few  parts  from  the  available  samples . 
The  steps  by  which  the  failure  points  are  estimated  should  be  documented  in  the 
characterization  report.  Because  the  definition  of  parametric  failure  depends 
on  the  Intended  device  application,  the  estimated  failure  points  will  only  be 
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approximately  correct.  They  will  nevertheless  be  useful  for  adjusting  the 
irradiation  increments  and  the  total  fluences  to  match  the  expected  device 

ormance .  The  irradiation  increments  and  the  total  fluences  selected  on  the 
basis  of  the  estimated  failure  points  should  be  listed  in  the  test  plan.  After 
the  characterization  measurements  are  completed,  device  failure  points  can  be 
more  accurately  defined. 

5. 1.8. 6  Gradual  degradation.  For  devices  which  degrade  gradually  as  they 
approach  failure,  it  will  generally  be  useful  to  measure  post-irradiation 
performance  characteristics  as  a  function  of  accumulated  radiation  fluence  or 
ionizing  radiation  dose  because  such  results  can  be  used  to  interpolate  or,  in 
some  cases,  extrapolate  the  part's  performance  to  radiation  levels  other  than 
those  used  in  the  tests .  For  such  devices ,  the  number  of  irradiations  and  the 
radiation  increments  should  be  made  commensurate  with  the  expected  failure 
levels .  It  is  worth  noting  that  a  gradual  approach  to  failure  may  not  always  be 
monotonic;  sometimes  a  parameter  may  first  increase  with  radiation  fluence  and 
then  subsequently  decrease.  This  behavior  means  that,  in  the  MIL  STD  system  and 
for  the  radiation  levels  shown  in  TABLE  1  (Section  4.2),  a  part  tjrpe  that  has 
been  qualified  at  a  higher  level  may  not  meet  the  same  RHEPL  at  a  lower  level. 
Q'*®llfication  at  a  hi^er  level  should  therefore  not  mean,  automatically,  that 
the  part  is  also  qualified  at  a  lower  level  unless  care  is  taken  to  select  the 
RHEPL  so  that  it  does  not  show  less  degradation  at  the  higher  level. 

3. 1.8. 7  Abrupt  failure.  Some  devices  give  little  or  no  indication  that  a 
radiation  failxire  point  is  being  approached,  until  they  fall  abruptly;  usually 
abrupt  failures  occur  as  loss  of  functionality.  The  claim  has  sometimes  been 
made  that  there  is  usually  some  gradually  deteriorating  parameter  which  can  be 
used  as  a  predictor  of  abrupt  failure  (e.g.  a  flip-flop  circuit  falling 
functionally  when  the  fanout  capability  of  the  circuit,  as  Indicated  by  the  sink 
current,  degrades  past  a  certain  point).  As  a  practical  matter,  however,  a 
device  must  be  considered  to  fall  abruptly  when  there  is  no  feasible  way  to  pin¬ 
point  its  failiure  level  by  means  of  interpolation.  An  estimated  failure  level 
is  also  important  for  such  devices  and  should  be  used  for  selecting  the  number 
and  Increment  value  for  the  irradiations  such  that  the  abrupt  failure  point  will 
be  narrowly  bracketed.  Whenever  possible,  the  most  desirable  measurement  is  to 
monitor  the  device  while  it  is  being  irradiated  to  determine  the  exact  stress 
level  where  failure  occvirred.  Again,  the  estimated  failure  point  will  have  to 
be  based  on  previotis  data  or  on  measurements  on  a  limited  sample  size  before  full 
scale  characterization  measurements  are  made. 

5. 1.8. 8  Optimum  use  of  radiation  facilities.  Such  factors  as  the  number 
of  devices  which  can  be  irradiated  at  one  time,  the  length  of  time  required  for 
the  irradiation,  and  the  length  of  time  required  to  complete  post— irradiation 
measurements  should  be  considered  when  an  irradiation  schedule  is  being  planned. 
The  selected  irradiation  schedule  should  then  be  itemized  in  the  test  plan. 

5. 1.8. 8.1  Selection  of  radiation  levels.  For  each  type  of  radiation  and 
for  each  estimated  device  failtire  level,  a  series  of  irradiations  should  be 
selected  so  that  data  is  obtained  most  efficiently.  The  lowest  level  of 
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radiation  should  be  below  that  which  produces  any  failures.  In  general  the 
subsequent  irradiation  increments  will  be  sizeable  fractions  of  the  failure 
level.  If  significant  non-linear  behavior  is  produced,  however,  then  the 
location  of  the  non— linear  region  must  be  estimated  and  the  radiation  increments 
should  be  made  smaller  than  they  are  outside  of  it.  The  highest  radiation  level 
should  be  high  enough  so  that  all  of  the  tested  parts  will  fail.  This  selection 
will  not  only  allow  the  device  specification  to  be  written  for  the  maximum 
capability  of  the  part  type,  should  that  be  necessary,  but  will  provide 
information  on  how  close  the  part  is  to  failure  for  a  given  system  application. 

5. 1.8. 8. 1.1  Selection  of  radiation  levels  and  abrupt  failure.  For  step 
stress  measurements  and  the  presence  of  abrupt  failure,  the  proper  selection  of 
radiation  stress  levels  is  of  particular  importance  because  a  bad  choice  can 
compromise  the  measurements.  The  spacing  between  stress  levels  should  be  as 
small  as  feasible.  At  the  very  least,  care  should  be  taken  to  assure  that  the 
spacings  are  small  compared  to  the  estimated  standard  deviation  of  the  failure 
fluences  for  the  parts  to  be  tested.  An  estimate  of  this  standard  deviation  may 
not  be  easy  to  obtain.  As  a  first  approximation,  data  on  similar  parts  may  be 
used.  If  there  is  no  suitable  data,  smaller  than  expected  spacing  may  be 
necessary  for  the  first  few  parts  and  then,  if  costs  can  be  lowered  thereby,  the 
spacing  for  the  rest  of  the  parts  can  be  adjusted  according  to  the  measured 
standard  deviation.  For  a  complete  characterization,  the  highest  stress  level 
mxist  be  large  enou^  to  drive  all  the  tested  parts  to  failure.  Very  wide  stress 
levels  may  require  an  extra  large  sample  size.  Less  than  four  stress  levels  are 
unacceptable  if  abrupt  failure  of  devices  is  considered  a  possibility. 

5. 1.8. 9  Efficient  use  of  test  samples.  The  costs  of  radiation  testing  and, 
for  some  part  types,  the  cost  of  the  parts  themselves  both  provide  strong  reasons 
for  keeping  the  nvunber  of  test  parts  as  low  as  possible.  In  these  circumstances 
it  is  essential  that  the  experiments  be  specifically  planned  so  that  the  maximum 
amotmt  of  useful  Information  is  obtained  from  the  sample  size  that  is  available . 
Whenever  possible ,  preparations  for  the  actual  tests  should  be  conducted  on  spare 
and  perhaps  less  expensive  parts  which  are  not  part  of  the  final  sample. 

5. 1.8. 9.1  Selection  of  exposure  sequences.  Because  most  characterization 
measurements  are  destructive,  a  single  device  is  exposed  to  only  one  radiation 
environment  and  the  exposure  sequence  simply  proceeds  from  low  levels  to 
functional  or  parametric  failure.  The  cases  where  one  device  can  be  used  for  two 
radiation  environments  are  those  when  the  dose  rate  upset  threshold  measurement , 
which  can  be  non  damaging,  is  made  first  and  is  followed  by  neutron,  gamma  ray, 
or  SEU  tests.  In  these  cases  care  must  be  taken  to  ensure  that  the  accumulated 
ionizing  radiation  dose  in  the  dose  rate  testing  has  not  appreciably  changed  the 
characteristics  of  the  device.  The  use  of  one  device  for  two  radiation 
environments  is  not  recommended.  It  should  be  considered  only  when  sample  costs 
are  so  high  that  every  possible  way  of  conserving  sample  size  mvist  be  used.  Time 
dependent  effects  should  be  considered  when  a  series  of  exposures  is  being 
planned. 


36 


5.1.9  Radiation  Exposures .  Radiation  exposures  should  be  made  In 
accordance  with  the  test  plan  and  should  contain  at  least  the  Information  listed 
in  TABLE  8  (Section  5. 1.8. 4).  Initial  test  system  checkout,  e.g.  of  test 
equipment  operation,  dosimetry,  etc. ,  should  be  accomplished  with  parts  which  are 
less  costly  and  readily  available;  one  possibility  is  to  use  parts  of  the  same 
type  as  the  sample  but  selected  to  a  less  stringent  visual  criterion.  The  use 
of  a  commercial  equivalent  device  type  can  also  be  considered  but  such  use  will 
need  to  be  validated  experimentally. 

5. 1.9.1  Neutron  exposures.  Neutron  exposures  can  be  made  either  at  fast 
burst  reactors  or  at  water  moderated  reactors.  In  either  case,  the  dosimetry 
practices  at  the  selected  facility  should  be  checked  to  make  sure  that  a  valid 
1  MeV  displacement  damage  (Si)  equivalent  fluence  can  be  obtained  (ref.  ASTM 
standard  E722-85).  Test  method  1017  of  MIL  STD  883,  entitled  "Neutron 
Irradiation”  can  be  used  as  a  guide  for  the  exposure  procedtire.  Short  term  (of 
the  order  of  seconds)  annealing  effects  do  not  have  to  be  evaluated  MIL  STD  part 
types. 

5. 1.9. 2  lonizine  radiation  dose  (total  dose)  exposures.  Test  method  1019 
of  MIL  STD  883,  entitled  "Steady  State  Total  Dose  Irradiation  Procedure," 
specifies  either  Co-60  or  an  electron  beam  as  the  radiation  source  to  be  used  for 
ionizing  radiation  dose  testing.  Ionizing  radiation  testing  can  also  be 
performed  with  ■  Cesitjm-137  sources  and  with  low  energy  X-ray  sources.  If  a 
source  other  than  Co-60  is  used,  correlation  measurements  must  be  performed  to 
effect  a  comparison  with  Co-60.  High  energy  electron  Linacs  are  not  recommended 
because  of  the  possibility  that  the  displacement  damage  which  high  energy 
electrons  produce  may  Interfere  with  the  measurements.  Flash  x-ray  sources 
which,  typically,  deliver  the  dose  at  a  very  hi^  dose  rate,  are  not  recommended 
because  the  results  may  be  difficult  to  compare  with  Co-60.  Low  energy  x-ray 
sources  are  not  recommended  for  characterization  measurements  unless  a  valid 
correlation  to  Co-60  is  established. 

5 . 1 . 9 . 3 .  Bias  must  be  applied  on  the  device  during  the  Irradiations  and  the 
bias  conditions  must  be  in  accordance  with  the  test  plan.  If  measurements  are 
to  be  made  "in  flux,”  then  the  test  plan  should  specify  these  conditions  as  well. 
Because  time  dependent  effects  can  be  very  important  for  ionizing  radiation  dose , 
a  complete  time  history  of  the  irradiations  and  the  post-radiation  or  in-flux 
measurements  should  be  recorded.  This  time  history  should  include  the  start  and 
stop  times  of  both  the  irradiations  and  the  measurements  as  well  as  the  rate  at 
which  the  ionizing  radiation  dose  was  delivered.  A  pre-test  evaluation  of  the 
time  dependent  effects  may  be  also  be  necessary  to  make  sure  the  times  used  for 
the  irradiations  and  the  measurements  will  give  meaningful  results. 

5. 1.9. 4.  Transient  ionization  (dose  rate)  exposures  and  measurements .  High 
energy  electron  Linacs  are  the  facility  of  choice  for  dose  rate  upset  measure¬ 
ments.  If  Flash  x-ray  machines  are  used,  extra  care  is  required  with  the 
dosimetry  to  make  results  obtained  at  one  facility  comparable  to  those  obtained 
at  another.  MIL  STDS  1021  on  "Dose  Rate  Threshold  for  Upset  of  Digital 
Microcircuits”  and  1023  on  "Dose  Rate  Response  of  Linear  Microcircuits"  may  be 
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used  as  guides.  MIL  STD  1020  on  "Radiation  Induced  Latchup  Test  Procedure"  is 
vinder  revision,  principally  because  it  does  not  adequately  treat  the  case  of 
latchup  "windows".  It  can  be  used  for  general  guidance  provided  that  no 
interference  is  expected  from  this  "window"  problem. 

5. 1.9. 5  Single  event  unset  irradiations  and  measurements.  Single  event 
upset  (SEU)  measurements  are  sufficiently  complex  so  that  they  should  be  made  in 
collaboration  with  one  of  the  several  groups  in  the  U.S.  that  are  making  such 
measurements  routinely.  A  full  characterization  measurement  for  single  event 
upsets  or  latchup  produced  by  heavy  ions  requires  that  the  number  of  upsets  or 
the  latchup  in  a  particular  device  be  measured  as  a  function  of  the  linear  energy 
transfer  (LET)  of  the  ion.  Heavy  ion  irradiations  are  usually  performed  at 
Tandem  Van  De  Graaff  accelerators  but  may  also  be  performed  at  other  high  energy 
heavy  ion  accelerators  (ref.  ASTM  standard  F1192  entitled:  Standard  Guide  for  the 
Measurement  of  Single  Event  Phenomena  Induced  by  Heavy  Ion  Irradiation  of 
Semiconductor  Devices).  For  single  event  upsets  induced  by  protons,  a  measurement 
of  the  number  of  upsets  per  unit  fluence  at  a  proton  energy  of  60  MeV  or  hi^er 
should  be  adequate  as  an  input  data  point  to  a  theoretical  model  that  can  then 
be  used  for  estimating  the  total  ntimber  of  device  upsets  to  be  expected  in  some 
particular  proton  environment,  (ref.  Bendel  and  Petersen,  1983)  High  energy 
protons  for  SEU  tests  can  be  obtained  from  a  number  of  cyclotron  accelerators. 
Laser  and  Callforniuifr-252  techniques  are  being  studied  but  have  not  yet  advanced 
to  the  point  where  they  can  be  used  routinely  for  SEU  characterization 
measurements . 

5.1.10  Post-Irradiation  Measurements  and  Data  Examination.  The  parameters 
which  will  be  measured  after  irradiation  and  the  measurement  conditions ,  and  the 
equipment  if  possible,  should  be  the  same  as  those  for  the  pre— irradiation 
measurements.  TABLE  3  (Section  5.1.5)  and  TABLE  5  (Section  5. 1.7. 5.1)  list  the 
information  required.  The  test  plan  will  contain  the  actual  parameters  to  be 
measured  and  the  corresponding  operating  conditions.  However,  the  post- 
irradiation  measurement  procedure  will  have  to  take  into  account  the  possibility 
of  time  dependent  effects.  In  the  case  of  ionizing  radiation  dose,  time 
dependent  effects  can  be  especially  serious  and  must  be  evaluated  for  the  device 
being  tested.  MIL  STD  Method  1019  can  be  used  as  a  guide  until  an  improved 
method  for  taking  time  dependent  effects  into  account  is  developed.  Bias  must 
be  applied  during  the  irradiation.  Bias  oiay  not  be  necessary  at  all  times  after 
the  irradiation  but  the  bias  conditions  and  other  measurement  conditions  and 
time  intervals  should  be  as  consistent  as  possible  from  one  run  to  another.  Bias 
or  shorting  conditions  on  the  device  between  the  end  of  a  radiation  exposure  and 
the  start  of  electrical  measurements  should  be  recorded. 

Again,  the  measurement  results  should  be  examined  for  bad  devices  or 
outliers  and,  appropriate  action  taken  if  necessary.  This  examination  is 
important  in  preventing  bad  data  from  being  included  in  the  characterization 
report  and,  possibly  later,  from  entering  a  data  base. 

5.1.11  Data  recording.  TABLE  9  gives  a  summary  of  the  recording 
requirements  for  the  test  plan. 
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TABLE  9.  Svunmary  of  data  to  be  recorded  in  the  test  plan. 

PRE-RAD  MEASUREMENT  CONDITIONS:  Items  in  Table  4. 

IRRADIATION  CONDITIONS:  Items  in  Table  8. 

POST-RAD  MEASUREMENT  CONDITIONS:  Items  in  Table  4. 

DATA  RECORDING:  As  in  Tables  5,  Table  6,  and  Table  7. 


5.1.12  Performance  Characterization  Report.  An  essential  part  of  any 
characterization  measurement  program  is  to  document  all  the  measurement  results 
completely  enougih  so  that  another  investigator  can  use  the  results  without  having 
to  consult  the  ‘originator.  For  a  variety  of  reasons,  time  and  funding  being 
principal  among  them,  this  task  is  often  not  performed  adeqxiately.  The 
importance  of  the  documentation  task  ,  however,  cannot  be  overemphasized.  The 
hope  here  is  that  if  the  documentation  task  is  made  part  of  the  test  plan  so  that 
it  must  be  included  in  any  checkoff  list  or,  perhaps,  in  the  contract  data 
requirements  list  that  is  part  of  all  government  contracts,  then  the  results  of 
characterization  measurements  will  be  more  available  to  other  users  in  the  future 
than  they  have  sometimes  been  in  the  past. 

5.1.12.1  Standard  data  format.  The  characterization  report  should  use  the 
standard  data  format  discussed  in  Section  5. 1.7. 5.  The  recorded  data  should 
include  the  device  identity  information  listed  in  TABLE  3  (Section  5.1.5)  and  the 
test  conditions  listed  in  TABLE  4  (Section  5. 1.7. 2). 

TABLE  10  gives  a  summary  of  the  recording  requirements  for  the  characteriza¬ 
tion  report. 
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TABLE  10.  Summary  of  information  to  be  recorded  in  the  characterization 

report. 


DEVICE  IDENTITY:  Items  in  Table  3  and  corresponding  discussion 

PRE— RAD  MEASUREMENT  CONDITIONS :  Items  in  Table  4  and  corresponding 
discussion 

IRRADIATION  CONDITIONS:  Items  in  Table  8  and  corresponding  discussion 

POST-^RAD  MEASUREMENT  CONDITIONS:  Items  in  Table  4  and  corresponding 
discussion 

DATA  RECORDING:  As  in  Tables  5,  6,  and  7  and  corresponding  discussion 
FINAL  RESULTS:  Items  in  Table  9  and  corresponding  discussion 


5.1.12.2  Transfer  of  characterization  data  to  ERRIC.  To  make  as  much 
radiation  response  data  as  possible  available  to  U.S.  users,  the  Defense  Nuclear 
Agency  supports  a  data  bank  for  such  data  at  the  Electronic  Radiation  Response 
Information  Center  (ERRIC)  operated  by  the  Kaman  Sciences  Corporation.  ERRIC  Is 
actively  collecting  data  from  other  data  banks  and  Is  very  much  Interested  in 
receiving  any  new  data  that  is  taken.  It  Is  strongly  recommended  here, 
therefore ,  that  the  characterization  data  which  Is  obtained  and  documented  In  the 
characterization  report  also  be  sent  to  ERRIC.  ERRIC  is  equipped  to  acquire  data 
from  computer  disks  as  well  as  In  printed  form.  The  address  and  telephone  number 
for  ERRIC  are: 

ERRIC 

Kaman  Sciences  Corporation 

816  State  Street 

P.O.  Box  1479 

Santa  Barbara,  CA  93102-1479 
(805)963-6484 

5.2  Calculation  of  Electrical  Parameter  End-Point  Limits.  The  previous 
sections  of  this  report  have  described  how  radiation  response  characterization 
measurements  should  be  made  so  they  can  be  used  for  defining  lot  acceptance  tests 
that  will  provide  hardness  assurance  In  future  procurement.  At  this  point  the 
assumption  Is  made  therefore  that  the  characterization  results  have  been  obtained 
and  the  discussion  turns  to  the  procedures  by  which  lot  acceptance  tests  should 
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be  defined.  The  basic  goals  here  are  to  select  the  sample  size  and  to  calculate 
post  irradiation  test  criteria  from  the  characterization  data  such  that  future 
radiation  lot  acceptance  tests  performed  against  these  criteria  will  meet  a 
selected  objective.  T3rpically,  the  selected  objective  will  be  a  desired  lot 
acceptance  probability  and  confidence  level. 

It  is  worth  noting  that  the  methods  described  here  are  not  limited  just  to 
radiation  hardness  assurance  but  can  be  used  for  the  more  general  problem  of 
calculating  end-point  limits  to  meet  a  selected  lot  acceptance  objective.  They 
may  be  of  use,  therefore,  to  any  manufacturer  of  parts  who  is  interested  in 
quantifying  the  lot  acceptance  results  he  can  expect  as  a  function  of  the  end¬ 
point  limit  or  specification  value  he  selects  for  a  given  parameter. 

5.2.1  Lot  Acceptance  Tests.  Lot  acceptance  tests  are  based  on  testing  a 
sample  of  parts  after  they  have  been  irradiated  and  determining  whether  the 
sample  of  parts  passes  or  fails  some  specified  criterion.  If  a  variables  test 
is  being  used  then  the  measured  mean  value  and  standard  deviation  of  the  post- 
irradiation  parameter  values  will  be  compared  to  the  specified  criteria  to 
determine  whether  the  test  was  passed  or  failed.  If  an  attribute  test  (such  as 
an  LTPD  test)  is  being  used,  then  specified  RHA  end-point-limits  (RHEPL)  and, 
possibly,  functionality,  will  be  used  for  determining  whether  each  part, 
individually,  has  passed  or  failed  the  test.  Changes,  or  "deltas"  in  a  parameter 
may  also  be  used  as  a  RHEPL. 

5. 2. 1.1  Attribute  lot  acceptance  tests  in  Military  standard  procurement. 
Because  military  standard  procurement  rely  almost  exclusively  on  attribute  tests, 
the  discussion  here  mostly  addresses  this  type  of  test.  It  should  be  recognized, 
however,  that,  for  the  same  number  of  samples  tested,  radiation  lot  acceptance 
tests  based  on  variables  measurements  will  usually  give  higher  quality  results. 
Their  principal  disadvantages  are  that  they  are  somewhat  more  complex  and  costly 
and  require  assumptions  about  the  probability  distribution  to  which  the  parts 
belong.  Attribute  tests  such  as  the  lot  tolerance  percent  defective  (LTPD)  tests 
used  in  the  NIL  STD  system  have  the  advantages  that  they  require  less  documenta¬ 
tion  of  test  results,  do  not  require  as  much  training  for  test  personnel,  and  do 
not  depend  strongly  on  asstuiq>tions  about  the  probability  distributions  to  which 
the  tested  parts  belong.  The  relative  merits  of  these  two  types  of  tests  were 
discussed  in  some  detail  in  the  Introduction. 

5. 2. 1.2  End  point  limits  for  LTPD  tests.  FIGURE  4  shows  the  steps  to  be 
taken  for  calcxilatlng  end-point  limits  for  LTPD  lot  acceptance  tests.  TABLE  11 
lists  these  steps  in  finer  detail  as  a  step-by-step  check  off  procedure. 
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FIGURE  4.  Steps  for  calculating  end~point  limits  for  lot  acceptance 

tests. 


42 


TABLE  11.  Step  by  step  procedure  for  calculating  end-point  limits. 


Check-Off  Item 

1. 

Select  device  type  and  manufacturers 

2. 

Obtain  characterization  data  and  documentation 

3. 

Select  desired  lot  acceptance  probability,  confidence  level, 
sample  size  for  LTPD  tests 

and 

4. 

Select  parameters  to  be  tested  and  test  conditions 

5. 

Begin  data  examination 

6. 

Check  data  for  sufficiency 

7. 

Check  for  outlying  devices 

8. 

Check  for  outlying  lots 

9. 

Discatd  outliers 

10. 

Check  data  for  abrupt  failures 

11. 

If  abrupt  failures  have  a  significant  likelihood,  parameter  end  point 
limits  usually  cannot  be  calculated 

12. 

Check  for  unexpected  functional  dependencies 

13. 

Check  for  systematic  or  non- random  effects 

14. 

Check  for  correlations  with  respect  to  time  of  manxifacture 

15. 

Check  for  unexpected  magnitudes  of  lot-to-lot  variations 

16. 

Check  for  bad  data  points 

17. 

Correct  or  discard  bad  data  points 

18. 

Check  data  for  sufficiency 

19. 

Recast  the  data  to  the  specified  conditions 

20. 

Determine  within- lot  and  lot-to-lot  variations 

21. 

If  within- lot  variations  are  larger  than  lot-to-lot,  use  case  1 
analysis  method  to  calculate  the  R^  end-point  limits  (RHEPL) 

22. 

If  within- lot  variations  are  smaller  than  lot-to-lot,  use  case  2 
analysis  method  to  calculate  the  RHEPL. 

23. 

Incorporate  the  RHEPL  and  LTPD  lot  acceptance  test  requirements 
the  device  specification. 

into 
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5. 2.1.3  Statistical  uncertainties  associated  with  lot  acceptance  testing. 
The  lot  tolerance  percent  defective  tables  In  MIL-M  38510  and  other  MIL  STD 
documents  are  based  on  statistical  formulas  for  sampling  with  replacement,  l.e. 
the  sample  of  parts  used  for  the  tests  Is  replaced  Into  the  lot  of  parts  from 
which  It  was  drawn.  Because  radiation  tests  are  damaging,  the  parts  tested  are 
not  replaced  Into  the  lot.  The  statistical  Implications  of  this  fact  can  be 
complex  and  are  beyond  the  scope  of  this  docvunent.  In  general,  statistical 
uncertainties  are  reduced  when  the  lot  acceptance  sample  size  Is  large  and  are 
further  reduced  If  the  lot  acceptance  history  Is  known.  The  least  desirable 
situation  occurs  when  only  a  single  lot  has  been  tested  and  when  the  sample  size 
Is  only  a  few  parts,  say  4  or  less.  In  this  latter  case  It  will  be  difficult  to 
estimate  what  the  performance  characteristics  of  the  passed  lot  will  be.  The 
recommendation  Is,  therefore,  that  lot  acceptance  tests  based  on  4  parts  or  less 
should  not  be  used.  For  small  sample  sizes.  Information  about  the  uniformity  of 
the  lots  is  also  very  Important.  If  a  lot  were  perfectly  uniform,  for  example, 
then  testing  a  single  part  would  be  adequate.  If  the  uniformity  of  each  lot  is 
high  then,  during  the  lot  acceptance  tests,  most  of  the  time,  either  all  the 
parts  pass  or  they  all  fall.  For  such  uniform  lots,  a  2/0  or  a  4/0  test 
(presently  specified  In  the  group  E  tests  of  method  5005  In  MIL  STD  883)  may  be 
able  to  give  some  confidence  that  the  passed  lots  are  of  hl^  quality.  If  lots 
are  mostly  being  rejected  becaxxse  1  part  out  of  the  sample  Is  falling,  then  the 
passed  lots  must  be  considered  to  be  of  low  quality. 

5.2.2  Definition  of  RHA  End-Point  Limits  (RHEPL).  The  parameter  values 
against  which  the  post  radiation  performance  of  a  part  will  be  compared  to 
determine  If  the  part  has  passed  or  failed  the  test  have  been  termed  RHA  end¬ 
point  limits  (RHEPL) .  These  RHEPL  values  are  included  In  the  procurement 
specification  and  are  used  for  the  lot  acceptance  tests.  If  the  procurement 
system  uses  a  qualified  parts  list,  as,  for  example,  the  present  MIL  STD  system 
does,  then  these  RHEPL  may  also  be  used  for  part  qualification.  The  discussions 
which  follow  explain  what  decisions  must  be  made  before  the  RHEPL  can  be 
calculated  and,  once  the  required  decisions  are  made,  how  they  can  be  calculated 
from  the  characterization  measurements.  If  the  part  can  be  purchased  against  any 
one  of  several  radiation  levels,  as  for  example  the  four  Ionizing  radiation  dose 
levels,  M,  D,  R,  and  H,  shown  for  MIL  STD  parts  In  TABLE  1  (Section  4.2),  then 
a  separate  RHEPL  must  be  calculated  for  each  radiation  level. 

A  device  can  cease  to  f\inctlon  as  a  result  of  a  radiation  exposure  and  will, 
obviously,  fall  the  test  being  conducted.  Such  failures  are  not  associated  with 
the  definition  of  an  end-point  limit.  Of  course,  the  fiinctlonal  failure  of  a 
part  will  count  as  a  failure  for  lot  acceptance  purposes. 

5. 2. 2.1  One  and  two  sided  limits.  A  parameter  used  for  determining  whether 
a  device  has  passed  or  failed  a  test,  may  be  bounded  by  maximum  and  minimum 
values,  l.e.  by  "two  sided"  limits,  or  by  either  a  maximum  or  a  minimum  value, 
l.e.  by  a  one-sided  limit.  Because  a  one-sided  limit  is  the  one  most  commonly 
used  In  radiation  response  testing,  only  such  limits  are  treated  here.  The 
mathematical  formulas  for  the  case  of  two-sided  limits  are  almost  identical  to 
the  one-sided  case  so  the  present  discussion  can  easily  be  extended  to  that  case 
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if  it  is  necessary  to  do  so. 


5.2.3  Critgrlfl  for  Calcvletine  RHA  End-Polnt  Limits  rRHEPL^ ■ 

5. 2. 3.1  S^J.ection  of  objectives  for  lot  acceptance  tests.  Meaningful  RHA 
end— point  limits  cannot  be  calculated  until  the  objectives  of  the  lot  acceptance 
testing  have  been  selected.  Simply  put,  if  the  object  of  the  tests  is  that  at 
least  90  percent  of  future  lots  should  pass  the  test,  the  end-point  limits  will 
be  less  stringent  than  if  the  object  is  that  at  least  80  percent  of  future  lots 
should  pass.  It  should  be  noted,  however,  that  the  statistical  uncertainties 
associated  with  the  relatively  small  sample  sizes  that  are  t3rpically  used  (tests 
usually  based  on  tens  of  devices  but  not  hundreds)  have  the  consequence  that, 
almost  regardless  of  where  the  RHEPL  is  set,  there  will  be  a  significant 
probability  that  a  good  lot  may  fail  or  that  a  bad  lot  may  pass  the  specified  lot 
acceptance  test.  The  real  constraints  on  selecting  an  objective  for  the  lAT  and 
on  calculating  the  corresponding  RHEPL,  therefore,  are  the  performance  of  the 
part  and  the  economic  costs  of  rejecting  lots.  Thvis  if  the  RHEPL  is  set  so  that, 
with  90%  confidence,  10%  of  future  lots  may  be  expected  to  pass,  then  the 
performance  of  the  lots  that  pass  may  be  very  good  but  the  costs  of  rejecting 
such  a  hl^  percentage  of  lots  may  make  the  part  too  expensive  to  use. 
Similarly,  if  the  end-point  limit  is  set  so  that  90%  of  future  lots  may  be 
expected  to  pass ,  then  the  cost  of  the  part  may  be  attractive  but  the  performance 
of  the  part  may,  not  as  good  as  may  be  desired.  The  objectives  of  the  lot 
acceptance  tests  are  thus  seen  as  a  trade— off  between  passing  the  largest 
percentage  of  lots  and  keeping  end-point  limits  which  will  make  the  performance 
of  the  part  desirable. 

5. 2. 3. 2.  Lot  relection  probability  and  choice  of  end  points.  In  the  data 
analysis  sections  which  follow,  these  guidelines  show  how  lot  rejection 
probability  depends  on  the  choice  of  end-point  limits.  They  should,  therefore, 
assist  manufacturers  or  specification  developers  in  selecting  end-point  limits 
which  best  meet  their  needs.  The  discussion  also  shows  that  lot  to  lot 
variability  in  radiation  response  can  present  serious  difficulties  to  end-point 
limit  selection.  For  this  latter  case,  which  is  common,  the  methods  recommended 
here  may  provide  a  better  assessment  of  the  risks  associated  with  different 
values  of  the  endpoint  limits  than  other  methods  which  have  been  used. 

5.2. 3.3.  Many  parameters  and  environments.  The  most  tractable  case  for 

calculating  end-point  limits  that  are  to  be  used  for  lot  acceptance  tests  is  the 
case  of  a  single  radiation  environment  and  a  single  device  parameter.  As  soon 
as  two  or  more  different  and  independent  test  parameters  are  involved  in  lot 
acceptance,  then  the  probabilities  of  passing  for  each  parameter  have  to  be 
higher  by  amounts  such  that  the  product  of  all  the  passing  probabilities  equals 
the  probability  of  passing  that  is  desired.  Thus  for  example,  if  two  different 
and  independent  parameters  are  specified  and  the  desired  lot  acceptance 
probabill^  is  0.9,  then  the  respective  passing  probabilities,  PI  and  P2,  have 
to  be  such  that  (P1)*(P2)  -  0.9.  ^ 

The  complexities  of  the  calculations  for  combined  radiation  environ- 
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ments  or  parameters  or  both  are  such  as  to  put  them  outside  the  scope 
of  this  document.  Unless  otherwise  noted,  therefore,  the  analysis 
examples  given  In  the  sections  which  follow  are  for  a  single  radiation 
environment  and  a  single  device  parameter. 

5. 2. 3. 3.1.  Many  parameters.  In  practice,  device  specifications  generally 
do  have  to  require  that  post  radiation  tests  be  made  on  several  parameters 
against  specified  end-point  limits.  The  recommendation  here  Is  that,  whenever 
possible,  the  characterization  data  be  used  to  select  the  most  sensitive  critical 
parameter  and  that  the  end— point  limit  be  calculated  for  that  parameter  to  give 
the  desired  lot  acceptance  probability.  The  end-point  limits  for  the  ocher 
relevant  parameters  then  must  be  calculated  to  give  sufficiently  high  passing 
probabilities  so  that  Chose  parameters  will  not  cause  a  significant  number  of  lot 
failures.  Methods  are  presently  being  developed  to  deal  with  the  case  when  two 
or  more  Independent  parameters  have  comparable  likelihoods  of  causing  failure  but 
they  are  not  yet  In  a  form  which  could  be  Included  here. 

5. 2. 3. 4  End  points  and  electrical  condlstlons.  It  Is  common  also  for  a 
device  specification  to  give  values  of  a  lot  acceptance  parameter  as  a  function 
of  Che  operating  condition  of  Che  device.  Thvis,  for  example,  end-polnt-llmlts 
stay  be  given  for  the  post  Irradiation  gain  of  a  transistor  as  a  fxinction  of  the 
collector  current.  It  can  then  happen,  because  the  calculations  are  statistical 
In  nature,  chat  different  mean  values  and  standard  deviations  In  the  character¬ 
ization  data  for  the  different  collector  currents  can  result  In  calculated  end- 
polnt-llmlts  that  do  not  behave  In  the  expected  way  as  function  of  the  collector 
current.  The  following  caution  Is  therefore  advised: 

Caution:  Data  deficiencies  and/or  statistical  effects  can  lead  to  end- 
polnt-llmlts,  calculated  as  a  function  of  some  device  parameter,  that 
do  not  behave  in  a  reasonable  way.  For  example,  for  a  given  neutron 
fluence,  the  calculated  RHEPL  for  transistor  gain  may  show  a  decrease 
with  Increasing  collector  current.  Instead  of  an  Increase  (In  the 
region  below  the  collector  ctirrent  at  which  the  gain  is  a  maximum) . 

It  Is  Important  therefore  that  the  calculated  results  be  subjected  to 
a  "sanity  check"  to  make  sure  that  "unphyslcal"  end-polnt-llmlts  are 
not  entered  Into  the  device  specifications.  In  general,  a  RHEPL 
should  be  a  smooth  function  of  bias  and  radiation  stress.  It  should 
be  noted  In  this  connection,  however,  that  a  gradvial  approach  to 
failure  nay  not  always  be  monotonic;  sometimes,  for  example,  a 
parameter  may  first  Increase  with  radiation  fluence  and  then  subse- 
qiiently  decrease. 

5. 2. 3. 5  Abrupt  failures .  If  abrupt  failures  are  expected  to  be  the 
dominant  failure  mode  at  the  radiation  levels  required  for  the  lot  acceptance 
tests ,  any  parameter  end-polnt-llmlts  that  are  used  should  be  calculated  so  that 
they  will  not  cause  a  significant  number  of  lot  failures.  This  situation 
resembles  that  of  the  several  parameters  discussed  in  Section  5. 2. 3. 3.  A  method 
for  estimating  end-point  limits  for  devices  which  suffer  abrupt  failure  has  been 
given  by  Namenson  and  Arlmura  (1985);  this  paper  Is  listed  In  Section  2.2.  In 
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practice,  even  though  lot  acceptance  tests  may  be  conducted  at  a  radiation  level 
where  some  abrupt  failures  may  occur,  the  subject  part  type  should  not  be  used 
In  a  system  application  If  the  specified  radiation  level  is  such  that  the 
likelihood  of  abrupt  failure  is  expected  to  be  significant;  a  derating  factor  of 
two  on  the  radiation  level  has  sometimes  been  used  but  a  more  accurate  value 
should  be  based  on  actual  fluence  to  failure  data. 

5. 2. 3. 6  Use  of  end-point  limits  for  system  design.  Strictly  speaking,  the 
end-point  limits  used  for  lot  acceptance  tests  do  not  guarantee  how  the  parts 
will  behave  in  a  system  exposed  to  a  specified  radiation  environment. 
Nevertheless,  systems  designers  frequently  use  the  RHEPL  as  the  starting  point 
for  derating  a  given  parameter  for  a  particular  system  application.  For  systems 
with  moderate  RHA  requirements,  the  derating  may  be  very  small  or  the  RHEPL  may 
even  be  used  directly  in  the  system  design.  For  systems  with  more  stringent  RHA 
requirements,  the  RHEPL  value  may  be  derated  significantly  or  may  be  set  so  that 
only  the  hardest  lots  are  accepted  for  system  production.  Thtis ,  if  a  part 
specification  is  being  developed  for  a  particular  system,  the  needs  of  the  design 
engineer  should  be  considered.  The  entire  topic  of  lot  acceptance  and  its  impact 
on  system  design  and  survivability,  while  very  much  deserving  of  detailed 
discussion,  is  beyond  the  scope  of  this  document. 

5. 2. 3. 7  An  asstimed  lot  acceptance  oblective.  The  many  factors  involved  in 
selecting  an  objective  for  the  lot  acceptance  tests  have  been  discussed  in 
section  5. 2. 3.1.  To  facilitate  the  data  analysis  discussions  which  follow,  the 
lot  acceptance  objective  is  here  taken  to  be  that,  with  90  percent  confidence, 
at  least  90  percent  of  futxire  lots  are  intended  to  pass.  This  objective  means 
specifically  that  the  end— point— limit,  which  determines  the  probability,  P,,  that 
a  single  part  will  pass  the  test,  must  be  set  so  that  the  probability  that  the 
entire  test  will  be  passed  is  90  percent.  This  particular  objective,  in  general, 
does  not  unduly  penalize  the  performance  specifications  for  a  device  nor  does  it 
place  ^mdue  requirements  on  the  amount  or  quality  of  characterization  data.  The 
assximptlon  of  a  specific  lot  acceptance  objective  makes  it  possible  to  give 
actual  numerical  values  for  the  examples  which  are  to  be  discussed.  The  formulas 
required  are  given  in  their  parametric  forms  so  that  calculations  can  be  made  for 
other  lot  acceptance  objectives. 

5.2.4  Data  Analysis.  Because  most  radiation  response  results  display 
significant  and  seemingly  random  variations,  the  methods  needed  to  analyze  such 
data  are  statistical  in  natxire.  In  addition,  most  radiation  testing  involves  the 
use  of  sample  parts  which  are  destroyed  by  the  tests,  so  the  analysis  techniques 
and  expectations  of  future  performance  (against  which  lot  acceptance  parameter 
end-point  limits  are  calculated)  must  be  based  on  statistics  of  sampling  without 
replacement.  The  sections  which  follow  discuss  general  analysis  principles  and 
their  applications  but  do  not  attempt  to  cover  all  the  data  analysis  variations 
that  can  occur  in  practice. 

5. 2. 4.1  Data  examination.  A  necessary  prerequisite  for  beginning  any  data 
analysis,  which  was  mentioned  previously  but  must  be  discussed  in  greater  detail 
again  because  of  its  crucial  importance ,  is  an  examination  of  the  data  to  ensure 
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that  the  data  do  not  contain  any  faulty  measurements  or  non-random  effects.  The 
data  should  be  examined  also  for  sufficiency.  Data  examination  Is  all  the  more 
Important  when  a  significant  amount  of  time  has  elapsed  between  the  characteriza¬ 
tion  measurements  and  the  data  analysis  or  when  the  person  or  organization 
undertaking  the  data  aiulysls  Is  different  from  the  persons  who  took  the  data  In 
the  first  place.  Some  of  the  most  common  reasons  for  suspecting  that  data  may 
be  bad  are  dlsctissed  below. 

5.2.4. 1.1  Outlying  devices.  It  Is  not  uncommon,  In  a  batch  of  measure¬ 
ments,  to  find  that  one  or  more  devices  out  of  the  sample  show  failure  fluences 
or  post-lrradlatlon  parameter  values  that  are  significantly  outside  the  values 
that  would  be  expected  from  the  mean  value  and  variance  for  the  rest  of  the 
sample.  (Outlying  devices  may  occasionally  be  found  also  In  the  pre-lrradlatlon 
measurements.  The  asstimptlon  here  Is  that  the  examination  of  the  pre-lrradlatlon 
data  has  eliminated  any  such  devices  from  the  characterization  sample.)  Although 
the  Identification  of  an  "outlier"  can  sometimes  be  difficult,  a  plot  of  the 
cumulative  probability  distribution  for  the  radiation  fluences  at  which  the 
devices  fall  (or  the  post-irradiation  parameter  values)  will  usually  show  that 
the  device  in  question  is  not  part  of  the  population  distribution  that  is 
characteristic  of  the  rest  of  the  sample.  A  more  detailed  discussion  of 
cumulative  probability  plots  may  be  found  In  textbooks  on  statistics  or  In 
Appendix  E  on  Statistical  Techniques  in  DNA  report  5910  listed  In  Section  2.1.1. 
The  condtlned  MIL  Handbook  entitled:  "  Total  Dose  and  Neutron  Hardness  Assurance 
Guidelines  for  Semiconductor  Devices  and  Microcircuits”  also  contains  an  appendix 
on  statistical  techniques.  A  discussion  of  how  to  identify  outliers  may  be  found 
In  ASTM  standard  E178— 75  entitled:  "Dealing  with  Outlying  Observations." 

5. 2. 4. 1.2  Outlying  lots.  In  this  slt\iatlon,  data  Is  available  for  a  number 
of  lots  and  an  examination  of  the  data  shows  that  one  or  more  of  the  lots  are 
showing  an  unusual  mean  value  or  standard  deviation.  Again,  a  cumulative 
probability  plot  can  be  made  of  the  data  to  evaluate  whether  the  suspect  lot  or 
lots  belong  to  a  different  population  of  lots. 

5. 2. 4. 1.3  Sufficiency  of  data.  The  first  step  In  preparation  for  data 
analysis  Is  to  see  If  there  Is  sufficient  data  for  deriving  end-point  limits. 
The  recommendation  In  these  guidelines  Is  that  the  characterization  sample  should 
consist  of  3  different  lots  taken  at  least  one  month  apart,  with  5  wafers  per  lot 
and  5  parts  per  wafer  taken  one  from  each  quadrant  and  one  from  the  center.  The 
minimum  sample  size  that  Is  recommended  is  5  different  lots  with  5  samples  per 
lot.  The  data  should  also  contain  a  range  of  values  and  radiation  stresses. 
Thus,  for  example,  If  step— stress  measurements  were  performed  and  all  the  parts 
failed  In  a  single  "bin"  (^ere  a  bln  Is  defined  as  the  Interval  between  one 
fluence  and  the  next  higher  fluence),  there  is  not  enough  data  for  an  analysis. 
If  It  happens  that  there  is  no  way  to  acquire  sufficient  data,  special  analysis 
techniques  may  be  required  and/or  some  worst  case  assumptions  may  be  needed  to 
supplement  the  data.  Sometimes  It  may  happen  that  the  number  of  lots  used  for 
obtaining  the  characterization  data  Is  not  given  In  the  data.  For  such  cases, 
a  method  exists  for  estimating,  from  the  data  itself,  what  the  effective  sample 
size  of  the  device  population  Is  and  this  effective  sample  size  can  be  introduced 
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into  the  end— point— limit  calculations  (see  ref.  A.  I.  Namenson,  1979,  in  Section 

2.2). 


5. 2. 4. 1.4  Abrupt  failures.  The  data  should  be  examined  to  see  if  abrupt 
failures  are  likely  at  the  radiation  levels  that  will  be  required  for  the  lot 
acceptance  tests.  If  abrupt  failures  are  expected  to  be  the  dominant  failure 
mode  at  these  radiation  levels,  then  the  mean  value  and  the  standard  deviation 
of  the  fluences  to  abrupt  failure  in  the  characterization  data  should  be  used  to 
estimate  the  radiation  level  at  which  the  part  can  be  used  safely.  The  fluences 
to  abrupt  failure  are  then  used  to  check  for  outlying  devices ,  outlying  lots ,  and 
the  sufficiency  of  the  data. 

5. 2. 4. 1.5  Unexpected  functional  dependencies.  If  a  parameter  value  does 
not  vary  smoothly  with  either  bias  conditions  or  radiation  dose,  the  data  may  be 
suspect . 

5. 2. 4. 1.6  Systematic  or  non-random  effects .  The  sampling  of  lots  and  parts 
must  be  a  truly  random  representation  of  future  lots  if  valid  lot  acceptance  end¬ 
points  are  to  be  determined.  It  pays,  therefore,  to  check  whether  the  data  is 
random  in  nature.  Some  typical  causes  of  non-random  effects  are  discussed  in  the 
sections  which  follow. 

5. 2. 4. 1.6.1  Correlations  with  respect  to  time.  Lot-to-lot  variations 
should  be  checked  to  see  if  they  correlate  with  time .  Exaotples  of  such 
correlations  are  an  average  shift  with  respect  to  date  of  manufacture  or  simply 
a  smooth  variation  with  respect  to  the  date  of  manufacture.  Usually,  a  look  at 
the  data  is  sufficient  to  ^termine  if  stich  an  effect  is  occurring.  However, 
more  rigorous  techniques,  such  as  time-series  correlations,  for  example,  exist 
which  can  check  whether  the  data  varies  smoothly  with  respect  to  time  or  has 
seasonal  variations. 

5. 2. 4. 1.6. 2  Unexpected  magnitudes  of  lot-to-lot  variations .  If  lot-to-lot 
variations  of  either  the  mean  value  or  the  standard  deviation  are  much  larger  or 
much  smaller  than  expected,  it  is  cause  for  concern.  If  a  number  of  lots  are 
very  similar,  then  it  may  be  that  the  sampling  over  lots  is  not  truly  random. 
There  may  be,  for  example,  a  single  diffusion  lot  involved  or  a  single  wafer. 
If  the  lots  were  produced  imder  total  process  control,  very  small  variations  may 
result  between  lots.  Then,  if  future  lots  will  come  from  the  same  process,  the 
characterization  data  is  valid.  However,  if  future  lots  will  not  all  come  from 
the  same  process,  then  the  data  is  not  valid. 

5. 2. 4. 1.6. 3  Other  systematic  effects.  If  the  data  comes  from  several 
sources,  it  can  vary  systematically,  i.e.  non-randomly,  according  to  the  source 
of  the  data.  Even  in  one  test  facility,  there  may  be  a  systematic  dependence  on 
test  cell,  operator,  dosimetry  equipment,  etc.  If  lots  come  from  several  plants, 
there  stay  be  a  systematic  dependence  on  which  plant  supplied  the  parts.  This 
type  of  effect  should  not  be  common;  it  is  mentioned  here  to  make  sure  that  is 
not  entirely  overlooked. 
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5. 2. 4. 4  Examples  of  data  analyses.  Depending  on  the  exact  nature  of  the 
data,  the  statistical  methods  that  should  be  used  to  analyze  the  data  and  to 
calculate  the  required  end-point  limits  range  from  relatively  simple  techniques 
to  some  that  are  still  under  development.  Examples  are  given  below  for  two  cases 
for  which  the  calculations  are  relatively  straightforward.  All  statistical 
quantities  are  assumed  to  obey  lognormal  probability  distributions,  i.e.  the 
logarithms  of  the  parameter  values  are  normally  distributed.  This  assumption  may 
be  considered  reasonable  because  many  of  the  parameters  that  are  of  interest  for 
radiation  effects  in  devices  have  been  observed  experimentally  to  obey  a 
lognormal  distribution.  In  any  case,  extensions  of  the  analysis  techniques 
discussed  below  can  be  made  to  other  types  of  probability  distributions. 

5. 2. 4. 4.1.  Parameters  and  delta-parameters.  For  convenience,  the  analyses 
Are  discussed  in  terms  of  the  value  of  the  parameter  in  question  after 
irradiation.  In  practice,  however,  it  is  often  useful  to  measure  the  changes, 
or  deltas,  in  a  parameter  value  produced  by  the  irradiation.  Again,  the 
extension  from  the  formulas  given  to  the  case  of  parameter  deltas  is  straightfor¬ 
ward.  It  should  be  noted  that  lot  acceptance  tests  based  on  parameter  deltas  can 
be  somewhat  more  costly  because  they  require  that  part  identity  be  maintained  and 
that  pre  and  post-irradiation  parameter  values  be  recorded.  They  should  not  be 
overlooked,  however,  because,  in  some  cases,  they  can  provide  better  hardness 
assurance  than  lot  acceptance  tests  based  just  on  the  post-irradiation  parameter 
value.  The  example  given  for  case  1  is  based  on  the  values  of  a  parameter;  the 
example  given  for  case  2  shows  how  deltas  may  be  used. 

5. 2. 4. 4. 2  Case  1.  Within-lot  variations  larger  than  lot-to-lot  variations . 
At  the  present  time,  case  1  is  an  unusual  result  and,  when  it  occurs,  may 
indicate  suspect  data.  Most  commonly,  lot-to-lot  variations  are  larger  than 
within-lot  variations  (discussed  here  as  case  2).  Case  1  is  being  discussed 
first,  however,  because  the  formulas  used  for  it  are  more  familiar.  In  addition, 
it  may  turn  out  to  be  true  in  the  future,  when  total  quality  control  procedures 
have  been  successfully  Implemented,  that  the  wafers  coming  off  of  a  production 
lot  will  be  quite  uniform  and  that  most  of  the  device  variations  will  be 
associated  with  variations  across  the  wafer.  This  case  will  apply  to  such 
product. 

5. 2. 4. 4. 3  Assumptions  for  case  1.  Specifically,  the  assumptions  made  here 
are  that  the  parameter  being  measured  shows  a  graceful  decrease  in  value  with 
increasing  radiation  stress  and  that  the  within-lot  variations,  or  standard 
deviations,  are  larger  than  the  lot-to-lot  variations.  A  further  assumption  is 
that  the  within-lot  variations  do  not  vary  greatly  from  one  lot  to  another.  If 
this  latter  assumption  does  not  hold,  then,  to  be  on  the  safe  side,  the  data 
should  be  analyzed  with  the  method  of  case  2.  If  the  above  assumptions  hold, 
then  the  group  of  lots  from  idiich  the  data  was  obtained  can  be  considered  as  a 
single  lot  and  the  total  nu8A>er  of  parts  drawn  from  all  the  component  lots  will 
be  the  sample  size  used  in  the  formulas  that  require  sample  size.  Let  this 
sample  size  be  n.  Now  assume,  as  a  convenient  example,  that  the  lot  acceptance 
test  will  require  that  11  parts  will  be  irradiated  to  a  radiation  level  PHI  (SPEC) 
and  that  all  parts  must  pass  for  the  lot  to  be  passed.  To  satisfy  the  lot 
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acceptance  objective,  stated  as  an  assumption  in  Section  5. 2. 3. 7,  that  with  90 
percent  confidence  at  least  90  percent  of  future  lots  should  pass ,  each  part  must 
have  a  probability  of  passing,  P,  such  that: 

(P)“  -  0.9 

This  equation  is  satisfied  if  P  -  0.99.  For  obvious  reasons,  P  is  often 
also  called  the  part  survivability. 

5. 2. 4. 4. 4.  Statement  of  problem.  Because  the  characterization  sample  was 
purposely  taken  so  as  to  represent  the  performance  of  future  lots  as  accurately 
as  possible,  the  problem  can  now  be  restated  as  follows;  what  is  the  parameter 
value,  RHEPL,  such  that,  with  90  percent  confidence,  fraction  P  of  future  parts 
have  a  parameter  value  greater  than  RHEPL  (for  a  parameter  that  decreases  as  the 
radiation  stress  increases)  after  being  irradiated  to  radiation  level  PHI  (SPEC)? 
In  response  to  this  question  RHEPL  is  given  by  : 

RHEPL  -  MEAN[  LN(PAR(PHI(SPEC) ) )  ]  -  Kti,(C,P,n)  *  STDEV[  LN(PAR)  ], 

where  MEAN[LN(PAR(PHI(SPEC)))]  is  the  mean  value  of  the  logarithms  of  the 
parameter  values  measured  after  irradiation  to  PHI (SPEC)  for  each  of  the  n 
devices,  STDEV(LN(PAR))  is  the  standard  deviation  of  these  same  logarlthsis,  and 
KTL(C,P,n)  is  the  one  sided  tolerance  limit  factor  for  confidence  C,  part 
survivability  P,  and  sample  size  n.  Note  that  a  mintis  sign  is  used  in  the 
equation  for  a  parameter  which  decreases  in  value  as  the  radiation  stress 
increases  and  a  plus  sign  is  used  for  a  parameter  that  Increases  in  value  as  the 
radiation  stress  increases .  The  one  sided  tolerance  limit  factors  may  be  found 
in  tables  in  the  statistical  references  previously  cited.  Some  of  the  most 
commonly  used  values  are  given  in  TABLE  12. 

The  equation  for  the  mean  value  of  the  logarithms  is: 

MEAN  [lN  (PAR(PHI(SPEC  )))]-  1  LN  (PAR^C  PHI  (SPEC  ))) 


where  PARj^ (PHI (SPEC))  denotes  the  value  of  the  parameter  after  irradiation  to 
PHI  (SPEC)  for  the  ith  device.  The  standard  deviation  of  these  logarithms  is 
given  by: 


STDEV  (LN  (PAR))- 


1 

n-1 


y  [LN  (PARi(PHI  (SPEC)) -MEAN  (LN  (PAR(PHI  (SPEC))))] 
1-1  •' 


1/2 
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TABLE  12.  One  sided  tolerance  limit  factors  - 


CONFIDENCE 

C  -  0.90 

N 

0.90 

P 

0.95 

0.99 

0.999 

0.9999 

3 

4.259 

5.311 

7.340 

9.651 

11.566 

4 

3.188 

3.957 

5.438 

7.129 

8.533 

5 

2.742 

3.400 

4.666 

6.111 

7.311 

6 

2.493 

3.091 

4.243 

5.555 

6.645 

7 

2.332 

2.894 

3.972 

5.202 

6.222 

8 

2.218 

2.755 

3.783 

4.955 

5.927 

9 

2.133 

2.649 

3.641 

4.771 

5.708 

10 

2.065 

2.568 

3.532 

4.628 

5.538 

11 

2.011 

2.503 

3.443 

4.514 

5.402 

12 

1.966 

2.448 

3.371 

4.420 

5.290 

13 

1.928 

2.403 

3.309 

4.341 

5.196 

14 

1.895 

2.363 

3.257 

4.273 

5.116 

15 

1.867 

2.329 

3.212 

4.215 

5.046 

16 

1.842 

2.299 

3.172 

4.164 

4.986 

17 

1.819 

2.272 

3.137 

4.119 

4.932 

18 

1.800 

2.249 

3.105 

4.078 

4.884 

19 

1.781 

2.228 

3.077 

4.042 

4.841 

20 

1.765 

2.208 

3.052 

4.009 

4.802 

21 

1.750 

2.190 

3.028 

3.979 

4.766 

22 

1.736 

2.174 

3.006 

3.952 

4.734 

23 

1.724 

2.159 

2.987 

3.926 

4.704 

24 

1.712 

2.145 

2.969 

3.903 

4.677 

25 

1.701 

2.132 

2.952 

3.882 

4.651 

30 

1.657 

2.080 

2.884 

3.794 

4.546 

35 

1.623 

2.041 

2.833 

3.729 

4.470 

40 

1.598 

2.010 

2.793 

3.678 

4.411 

45 

1.576 

1.986 

2.761 

3.638 

4.363 

50 

1.559 

1.965 

2.735 

3.605 

4.324 

60 

1.532 

1.933 

2.694 

3.552 

4.262 

70 

1.511 

1.909 

2.662 

3.513 

4.215 

80 

1.494 

1.890 

2.637 

3.482 

4.178 

90 

1.481 

1.874 

2.617 

3.456 

4.148 

100 

1.470 

1.861 

2.601 

3.435 

4.124 
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5. 2. 4. 4, 4.1  An  illustrative  example  of  an  RHEPL  calculation  for  case  1. 

For  this  example  the  following  assvunptlons  are  made: 

a)  Lot  acceptance  tests  will  be  performed  on  the  current  gain  of  2N2222 
transistors  after  irradiation  to  PHI (SPEC)  -  2.5*10^®  neutrons/cm^  (IMeV 
silicon  displacement  damage  equivalent).  The  collector  current,  bias 
voltages,  and  temperature  for  the  test  will  be  given  in  the  device 
specification. 

b)  An  11/0  lAT  test  will  be  used,  i.e.  11  devices  will  be  irradiated  to 
2.5*10^^  neutrons/cm^  and,  if  even  1  device  fails,  the  lot  will  not 
accepted.  (In  MIL  STD  practice,  if  one  part  out  of  the  first  11  fails, 
then  a  drawing  of  7  additional  devices  is  allowed  and  the  lot  passes  if 
there  are  no  further  failures;  this  extension  of  the  LTPD  test  will  not  be 
treated  here . ) 

c)  With  90  percent  confidence  (i.e.  C  »  0.90),  at  least  90  percent  of  the  lots 
should  pass;  this  assiimption  was  made  in  Section  5. 2. 3. 7  precisely  so  that 
examples  such  as  this  one  could  be  calculated.  This  assvtmptlon  together 
with  assumption  b)  means  that  P  must  be  0.99. 

d)  The  characterization  data  was  adequate  to  provide  post— irradiation  current 
gain  values,  at  the  required  collector  current,  bias  voltage,  and 
temperature  at  2.5*10^  neutrons/cm^.  This  data  is  shown  in  TABLE  13. 
Because  this  is  an  illxistrative  example,  data  is  shown  for  only  10  devices. 
In  actual  practice,  a  sample  size  of  10  should  not  be  considered  adequate 
to  support  the  calculation  of  an  end-point  limit. 

For  the  values  in  TABLE  13,  the  following  quantities  may  be  calculated: 
MEAM(LN(hpE(PHI(SFEC))))  -  4.591  and 
STDEV(LN(hn(PHI(SPEC))))  -  0.107. 

Assumptions  b)  and  c)  make  C  —  0.90  and  P  -  0.99.  The  size  of  the  characteriza¬ 
tion  sample  is  10.  For  C  -  0.90,  P  -  0.99,  and  n  -  10,  the  one  sided  tolerance 

limit  factor,  Kix,i  is  found  in  TABLE  12  to  be  3.532.  The  end-point  limit,  RHEPL, 

is  thus: 


RHEPL  -  4.591  -  3.532  *  0.107  -  4.213 

In  more  familiar  terms,  the  geometric  mean  value  of  the  current  gain  for  the  ten 
devices  in  TABLE  12  is  98.6,  the  plus  and  minus  one  standard  deviation  gains  are 
109.7  and  88.6  respectively,  and  the  end-  point  limit  for  the  gain  is  67.6  (- 
exp(4.213)) .  Lot  acceptance  tests  for  2N2222  transistors  would  then  be  performed 
in  the  future  by  irradiating  11  devices  from  each  lot  to  2.5E13  neutrons  per  cm^ 
and  requiring  that  each  device  out  of  the  11  have  a  post  irradiation  gain  greater 
than  67.6;  otherwise  the  lot  is  rejected. 
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TABLE  13.  Post-irradiation  data  for  ten  2n2222  transistors. 


PHI(SPEC)  —  2.5*10^^  neutrons/cm^  (1  MeV  Si  damage  equivalent) 


DEVICE  NUMBER 

hpjC PHI (SPEC)) 

ln(hFE(PHI(SPEC))) 

1 

108.8 

4.689 

2 

105.4 

4.657 

3 

93.7 

4.540 

4 

97.7 

4.582 

5 

102.6 

4.630 

6 

104.5 

4.650 

7 

84.1 

4.431 

8 

101.1 

4.616 

9 

112.5 

4.723 

10 

81.1 

4.396 

5. 2. 4. 4. 5  Case  2 .  Lot-to-lot  variations  larger  than  within  lot  variations . 
Case  2  is  applicable  when  within— lot  standard  deviations  are  small  compared  to 
lot— to— lot  variations.  In  this  case,  the  effective  sample  size  for  the  RHEPL 
calculations  is  the  number  of  lots  and  not  the  number  of  devices  used  for  the 
characterization  measurements.  An  approximate  analysis  method  for  this  case  was 
developed  by  I.  Arimura  and  A.  Namenson  and  piiblished  in  IEEE  Trans.  Nucl.  Scl. , 
NS-30,  4322,  Deceoiber  1983.  The  method  assvunes  that  the  parameters  follow  a 
normal  or  lognormal  probability  distribution  but  it  can  be  modified  for  a 
different  distribution  if  necessary.  Once  again  the  assumption  described  in 
Section  5. 2. 3. 7  is  made,  namely  that  the  object  of  the  lot  acceptance  tests  is, 
with  90  percent  confidence,  to  have  at  least  90  percent  of  lots  pass.  It  is 
assumed  also  that  an  11/0  LTFD  test  will  be  used  for  lot  acceptance.  This  last 
assumption  means  that,  again,  the  individual  part  survival  probability  of  the  lot 
being  tested  for  acceptance  must  be  P  -  .99. 

5. 2. 4. 4. 5.1  SanmUn^  for  multi-lot  analysis.  Ideally,  there  should  be  at 
least  5  lots  with  at  least  5  parts  in  each  lot.  If  the  within-lot  variations  are 
very  small  compared  to  the  lot-to-lot  variations,  the  requirement  on  having  at 
least  5  parts  per  lot  may  be  waived. 

5. 2. 4. 4. 5. 2  Steps  in  multi-lot  analysis.  The  following  steps  are  then  used 
in  the  calculation: 

A.  Let  the  number  of  lots  be  N  and  the  number  of  devices  in  each  lot  be  n^. 

For  the  Ith  lot,  calculate  the  mean  value  and  the  standard  deviation  of 


the  given  parameter.  Specifically,  for  the  Ith,  lot  the  quantities 
calculated  are  MEANi(PAR(RAD))  and  STDEVi(PAR(RAD) )  as  given  by  the 
following  equations: 


MEAN  I  [PAR  (RAD)] 


1  Z  PAH  „  (RAD) 


where  PARjjCRAD)  is  the  measured  value  of  PAR(RAD)  for  the  jth  device  in 
the  Ith  lot,  and 


STDEV  [PAR  (RAD)]- 


1 

ni-1 


ni 


1/2 


PAR  ij  (RAD)  -MEAN  i  PAR  (RAD)  ]  ^ 


B.  For  each  lot  multiply  the  standard  deviation  by  2.326  (see  explanation 
below)  and  add  it  to  the  mean  (addition  is  used  for  a  parameter  that 
increases  in  value  with  Increasing  radiation  level) .  This  gives  a 
limit: 

LIMi-  MEANi(PAR(RAD))  +  2.326  *  STDEVi(PAR(RAD) )  for  the  Ith  lot. 

C.  For  the  N  values  of  LIHi,  obtain  the  MEAN(LIM)  and  the  standard  deviation 
STDEV(LIM) . 

D.  Look  up  the  one-slded-tolerance-limit  factor  Kn,  for  90%  probability  and 
90%  confidence  and  for  a  sample  size  corresponding  to  N,  the  number  of 
lots. 

E.  Use  the  following  equation  to  obtain  the  desired  RHEPL: 

RHEPL  -  MEAN(LIM)  +  Kii,(C  -  .9,  P  -  .9,  N)  *  STDEV(LIM) 

5. 2. 4. 4. 5. 3  The  rationale  of  the  method  given  above  is  to  examine  the 
distribution  of  the  99  percentile  points  of  the  different  lots  and  treat  them  as 
a  normal  (or  lognormal)  distribution.  For  each  lot,  adding  MEAN(FAR)  -t-  2.326  * 
STDEV(PAR)  gives  the  best  estimate  of  the  99  percentile  point  because,  for  a 
standard  normal  distribution,  2.326  standard  deviations  above  the  mean  Includes 
99%  of  the  distribution.  The  KtL  factor  in  step  D  above  then  is  used  to  obtain 
an  estimate  that,  with  90%  confidence,  90%  of  the  lots  will  have  99%  of  their 
parts  within  the  calculated  RHEPL. 

5. 2. 4. 4. 5. 4  An  illustrative  example  of  an  RHEPL  calculation  for  case  2. 
The  following  example  was  taken  from  actual  data  on  voltage  shifts  (deltas) 
measured  after  an  ionizing  radiation  dose  of  400  KRAD.  (This  data  is  interesting 
for  other  reasons  as  well  and,  for  those  reasons,  will  be  discussed  in  greater 
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detail  in  APPENDIX  II.)  The  lots  do  not  all  have  5  parts  per  lot,  but  the 
within-lot  standard  deviations  are  so  small  compared  to  the  lot-to-lot  variations 
that  the  case  2  method  can  still  be  used.  The  data  in  this  example  is  treated 
as  a  normal  distribution  (instead  of  a  lognormal  distribution). 

The  values  shown  in  TABLE  14  yield  the  values  shown  in  TABLE  15. 


TABU!  14.  Post— irradiation  voltage  shifts  (deltas). 


PHI (SPEC)  -  400  KRADS 

LOT  NUMBER 

DEVICE  NUMBER 

vOLTAGE  SHIFT  (mV) 

1 

1 

-6.79 

1 

2 

-6.52 

2 

1 

-5.46 

2 

2 

-5.38 

3 

1 

-3.76 

3 

2 

-3.67 

3 

3 

-3.63 

4 

1 

-2.41 

4 

2 

-2.18 

4 

3 

-1.72 

5 

1 

3.25 

5 

2 

3.60 

5 

3 

3.81 

5 

4 

4.10 

5 

5 

4.23 

5 

6 

4.62 

6 

1 

7.26 

6 

2 

7.28 

6 

3 

7.38 

6 

4 

7.44 
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TABLE  15.  Calculated  values  for  each  of  the  6  lots  given  in  table  14. 


LOT  NUMBER 

MEANi(PAR(RAD)) 

STDEVi(PAR(RAD)) 

LIMi 

1 

-6.655 

0.1909 

■6 . 211 

2 

-5.42 

0.0566 

•5.288 

3 

-3.687 

0.0666 

•3.532 

4 

-2.103 

0.3513 

•1.286 

5 

3.935 

0.4859 

5.065 

6 

7.34 

0.0848 

7.537 

The  values  of  LIMj  in  TABLE  15  yield  following  values: 

MEAN(LIM)  -  -0.6192 
STDEV(LIM)  -  5.6715. 

Now  for  C  —  0.9,  P  —  0.9,  and  N  —  6,  Kj^  is  2.493.  The  value  for  the  RHEPL 
therefore  is: 

RHEPL - 0.6192  +  2.493  *  5.6715  -  13.520. 

Future  lot  acceptance  tests  should  therefore  be  performed  against  a  voltage  shift 
of  13.520  mV. 
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A  LISTING  OF  CORRESPONDING  ERRIC  AND  MIL-STD  SYMBOLS 

10  SCOPE 

This  appendix  is  to  establish  a  correlation  between  ERRIC  symbols  and  MIL- 
STD  symbols.  This  appendix  is  not  a  mandatory  part  of  the  handbook.  The 
information  contained  herein  is  intended  for  guidance  only. 

20  APPLICABLE  DOCUMENTS . 

This  section  is  not  applicable  to  this  appendix. 


30  A  LISTING  OF  CORRESPONDING 

ERRIC  AND 

MIL-STD  SYMBOLS 

ITEM 

ERRIC 

MIL-STD 

SYMBOL 

SYMBOL 

CURRENTS 

HIGH  LEVEL  INPUT  CURRENT 

IIH 

^IH 

LOW  LEVEL  INPUT  CURRENT 

IIL 

IlL 

OUTPUT  SHORT  CIRCUIT  CURRENT 

lOS 

los 

HIGH  LEVEL  SUPPLY  CURRENT 

ICC 

^CCH 

LOW  LEVEL  SUPPLY  CURRENT 

ICC 

^CCL 

EMITTER-BASE  CUTOFF  CURRENT 

lEBO 

Iebo 

COLLECTOR-BASE  CUTOFF  CURRENT 

ICBO 

IcBO 

CURRENT  FLOW  INTO  AN  INPUT  TERMINAL 

IIN 

IlH 

INPUT  OFFSET  CURRENT 

OFSTI 

Iio 

INPUT  BIAS  CURRENT 

IBIAS 

IjB 

ZERO  SCALE  CURRENT 

IZS 

Izs 

FULL  SCALE  CURRENT 

IFS 

Ips 
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ITEM 

ERRIC 

SYMBOL 

MIL-STD 

SYMBOL 

IMPEDANCES 

RESISTANCE 

R 

R 

INPUT  RESISTANCE 

RI 

OUTPUT  IMPEDANCE 

ZOUT 

Zo 

OUTPUT  RESISTANCE 

ROUT 

ON  RESISTANCE 

RON 

ON  RESISTANCE 

RDS 

Rds 

TIME 

HIGH-LOW  PROPAGATION  DELAY  TIME 

TPHL 

tpHL 

LOW-HIGH  PROPAGATION  DELAY  TIME 

TPLH 

tpLH 

REVERSE  RECOVERY  TIME 

TRR 

tia 

RISE  TIME 

TR 

tjLH 

FALL  TIME 

TF 

tjHL 

REJECTION  RATIOS 


COMMON  MODE  REJECTION  RATIO 

CM  RR 

CMrr 

POSITIVE 

POWER 

SUPPLY  REJECTION  RATIO 

PWSRRP 

+PSRR 

PS„ 

NEGATIVE 

POWER 

SUPPLY  REJECTION  RATIO 

PWSRRN 

-PSRR 

PS„ 
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item  ERRIC  MIL-STD 

SYMBOL  SYMBOL 

GAINS  AND  TRANSFER  RATIOS 

FORWARD  CURRENT  TRANSFER  RATIO  HFE  h™ 

POWER  GAIN  AP  Pg 

MAXIMUM  AUTOMATIC  GAIN  CONTROL  RANGE  MAGC  Agg 

OTHER 

NOISE  FIGURE  NF  Np 

SLEW  RATE  SL  RA  Sr 

NONLINEARITY  NL  Nj. 

POWER  SUPPLY  SENSITIVITY  PSSIFS  PssIrs 

BIT  ERROR  BERR  SIGMA-NL 
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EVALUATING  DATA  WITH  CUMULATIVE  PROBABILITY  PLOTS  ON  PROBABILITY  PAPER 

10  SCOPE 

This  appendix  deals  with  unfortunate  but  nevertheless  frequent  problems  of 
suspect  data.  Part  characterization  and  lot  acceptance  are  not  complete  without 
ascertaining  the  validity  of  the  data.  The  appendix  will  concentrate  on  graphic 
techniques  for  evaluating  data.  While  data  validation  is  important,  this 
appendix  is  not  a  mandatory  part  of  the  handbook.  The  information  contained 
herein  is  intended  for  guidance  only. 


20  APPLICABLE  DOCUMENTS. 

This  section  is  not  applicable  to  this  appendix.  All  referenced  documents 
appear  in  Section  2  of  the  text. 

30  INTRODUCTION 

30.1  The  importance  of  checking  and  validating  data  cannot  be  emphasized 
too  strongly.  Often,  probleois  of  faulty  equipment,  incorrect  asstmptions  and 
htunan  error  will  be  known  before  data  analysis  begins.  After  obviously  suspect 
data  have  been  removed,  subtle  data  problems  may  remain  whose  resolution  will 
depend  on  the  graphic  techniques  to  be  mentioned  here.  Such  problems  include 
deviations  from  the  assumed  probability  distribution,  systematic  effects  and 
outliers.  Though  the  methods  are  applicable  to  a  wide  range  of  situations,  this 
discussion  will  concentrate  on  using  probability  plots  to  interpret  test  data. 

40  WHAT  PROBABILITY  PAPER  IS  AND  WHY  IT  IS  USED 

40.1  Normal  probability  paper.  FIGURE  II-l  illustrates  three  different 
ways  of  depicting  a  normal  distribution.  The  upper  curve  is  the  differential 
probability  function  for  a  normal  distribution.  The  middle  curve  (the  so  called 
"S-curve")  is  the  cumulative  normal  distribution  (the  integral  of  the  upper 
curve)  plotted  in  linear  coordinates.  The  bottom  curve  is  the  middle  curve  with 
the  ordinate  distorted  so  that  the  S— Curve  becomes  a  straight  line.  When  test 
data  is  plotted  on  this  latter  scale,  the  eye  can  check  their  fit  to  an  assumed 
form  by  checking  their  fit  to  a  straight  line.*  Note  that  for  normal  statistics 


*  But  note,  however,  that  the  statistics  of  fitting  a  straight  line  to  the 
data  does  not  follow  the  usual  regression  analysis.  This  will  become 
clear  when  the  mechanics  and  mathematics  of  making  the  plots  are 
explained. 
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FIGURE  What  is  probability  paper?. 


^0.2  Different  kinds  of  probability  paper  exist  for  different  kinds  of 
probability  distributions  but  the  discussion  here  will  deal  almost  exclusively 
with  normal  and  lognormal  statistics .  Lognormal  probability  paper  is  the  same 
as  normal  probability  paper  except  that  the  abscissa  is  a  logarithmic  instead  of 
a  linear  scale. 

40.3  Obtaining  probability  paper.  Probability  paper  may  be  purchased  for 
most  of  the  commonly  applied  distributions  (e.g.  normal,  Weibull  etc.).  Computer 
made  plots,  require  calculating  ordinate  positions  from  functions  F(P)  where 
P  is  a  probability,  F(Y)  is  a  cumulative  probability  distribution  function  andF(p) 
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is  its  anti-function.  (That  is  F[F(Y)]-Y.)  Often  a  look-up  table  with 
interpolation  will  be  sufficient  to  generate  such  a  function.  For  the  normal 
distribution,  many  texts  tabulate  F(P)  as  Zp. 

50  MAKING  PROBABILITY  PLOTS  FROM  TEST  DATA 

50.1  The  Basic  Plot.  The  construction  of  plots  on  probability  paper  will 
be  explained  by  example.  TABLE  II— 1  shows  20  simulated  data  points  drawn  from 


TABLE  II- 1.  Ranking  data  &  preparing  for  plot  on  normal  probability  paper. 


Unranked  Data 

Ranked 

Data 

Rank  P  - 

Data 

Attribute 

Data 

Attribute 

m 

i/(N+l) 

FBARCP") 

0.2372 

ATTRIB.  00 

-2.5370 

ATTRIB.  03 

1 

0.0476 

-1.6684 

-1.3551 

ATTRIB.  01 

-1.6817 

ATTRIB.  07 

2 

0.0952 

-1.3092 

-0.6667 

ATTRIB.  02 

-1.3551 

ATTRIB.  01 

3 

0.1429 

-1.0676 

-2.5370 

ATTRIB.  03 

-0.9622 

ATTRIB.  10 

4 

0.1905 

-0.8761 

-0.1477 

ATTRIB.  04 

-0 . 8430 

ATTRIB.  16 

5 

0.2381 

-0.7124 

0.6332 

ATTRIB.  05 

-0.6667 

ATTRIB.  02 

6 

0.2857 

-0.5659 

2.0648 

ATTRIB.  06 

-0.5864 

ATTRIB.  17 

7 

0.3333 

-0.4307 

-1.6817 

ATTRIB.  07 

-0.4012 

ATTRIB.  13 

8 

0.3810 

-0 . 3030 

1.2512 

ATTRIB.  08 

-0.1477 

ATTRIB.  04 

8 

0.4286 

-0 . 1800 

1.2845 

ATTRIB.  09 

0.1326 

ATTRIB.  12 

9 

0.4762 

-0.0597 

-0.9622 

ATTRIB.  10 

0.2371 

ATTRIB.  00 

11 

0.5238 

0.0597 

0.5161 

ATTRIB.  11 

0.3928 

ATTRIB.  18 

12 

0.5714 

0.1800 

0.1326 

ATTRIB.  12 

0.5161 

ATTRIB.  11 

13 

0.6190 

0.3030 

-0.4012 

ATTRIB.  13 

0.6332 

ATTRIB.  05 

14 

0.6667 

0.4307 

0.9639 

ATTRIB.  14 

0.9639 

ATTRIB.  14 

15 

0.7143 

0.5659 

1.8462 

ATTRIB.  15 

1.1946 

ATTRIB.  19 

16 

0.7619 

0.7124 

-0.8430 

ATTRIB.  16 

1.2512 

ATTRIB.  08 

17 

0.8095 

0.8761 

-0.5864 

ATTRIB.  17 

1.2845 

ATTRIB.  09 

18 

0.8571 

1.0676 

0.3928 

ATTRIB.  18 

1.8462 

ATTRIB.  15 

19 

0 . 9048 

1.3092 

1.1946 

ATTRIB.  19 

2.0648 

ATTRIB.  06 

20 

0.9524 

1.6684 

a  normal  distribution  (col.  1)  together  with  their  attributes  (col.  2).  In  this 
case  the  attribute  is  the  string  "ATTRIB.  "  followed  by  the  sequence  of  the  point 
(Nos.  00  through  19).  Coltimns  3  and  4  of  the  table  are  the  points  rearranged  by 
rank  with  their  attributes  preserved.  Column  5  is  the  rank,  1,  of  the  ordered 
points  and  Column  6  is  the  probability  P  -  i/(N+l)  where  N  is  the  sample  size. 
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^In  this  case  N-20.)  For  computer  made  plots,  colvimn  7  gives  the  function 

F(P)  for  a  normal  distribution.  FIGURE  II-2  shows  the  resulting  graph.  Note 
the  fit  to  a  straight  line. 


SAMPLE  or  20  POINTS  DRAWN  FROM  STD.HORfl.  DISTRIBUTION 


ARBITRARy  UNITS 

FIGURE  II-2.  Sample  normal  probability  plot  of  20  points  -  simulated  data. 


50.2  Coloring"  The  Points.  The  points  may  be  portrayed  with  different 
colors  and  designs  according  to  their  attributes  so  that  any  systematic  effects 
in  the  data  would  become  apparent.  FIGURE  II-3  shows  the  plot  of  FIGURE  II-2 
with  the  points  colored  according  to  whether  their  first  digit  is  zero  or  one. 
Clearly  the  sequence  of  the  point  has  nothing  to  do  with  its  rank. 
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SMIPU  or  26  POINTS  PIMM  FKON  STl.NOffl.  DISTIIlVriON 


MUITIIAIiy  UNITS 


FIGURE  II-3.  Plot  of  figure  II— 2  with  coloring  and  confidence  lines. 


50.3  One-Sided  Tolerance  Limits.  One-sided  tolerance  lines  may  be  drawn 
on  the  plot  so  that  confidence  limits  may  be  read  directly  from  the  graph.  In 
FIGURE  II-3  the  confidence  lines  use  an  approximation  given  by  Natrella 
(referenced  in  Section  2.1.2  of  the  text).  If  S  and  M  are  respective  best 
estimates  of  the  s.d.  and  mean,  the  50%  confidence  line  is  very  nearly  a  straight 
line  with  slope  1/S  and  intercept  M  with  the  horizontal  50%  probability  line. 

50.4  Stet>-Stress  Measurements.  In  some  cases  the  values  will  be 
accompanied  by  experimental  uncertainty.  A  typical  kind  of  uncertainty  results 
from  step-stress  measurements  where  a  parameter  or  stress-to-failure  has  merely 
been  determined  to  be  between  upper  and  lower  bounds  defining  a  "bin."  In  such 
cases,  it  is  advisable  to  plot  the  center  the  bin  and  display  the  uncertainty  as 
a  horizontal  line  whose  full  width  is  the  bin  size.  A  graph  of  this  kind  gives 
a  quick  check  of  how  the  measurement  uncertainties  compare  with  the  s.d.  of  the 
distribution.  If  these  uncertainties  are  comparable  to  the  s.d.,  then  step- 
stress  analysis  may  be  necessary.  For  normal  distributions,  plus  and  minus 
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infinity  are  valid  bounds  for  a  step  stress  measurement  (e.g.  a  device  which 
survived  a  maximum  test  level)  and  plotting  such  bounds  requires  some  creativity. 
A  suggestion  is  to  display  the  finite  (lower  or  upper  bound)  and  use  an  arrow 
pointing  to  plus  or  minus  infinity.  Section  60.3.2.1  of  this  appendix  has  a 
figure  showing  a  step-stress  probability  plot. 

50.5  Step-Stress  Measurements  And  Staircase  Plots.  In  some  cases  - 
particularly  in  determining  stress  to  failure  using  a  step— stress  measurement  — 
more  than  one  part  may  fail  within  a  given  ”bin”.  If  the  ith  through  jth  parts 
all  failed  within  the  same  bin,  then  probabilities  i/(N+l)  through  j/(N+l)  should 
be  plotted  all  with  the  same  center  and  same  "error  bar” . 

50.6  Summary .  In  summary,  the  procedure  for  making  probability  plots  is: 

1.  Rank  the  points  preserving  their  attributes 

2.  Compute  probabilities  i/(N+l)  where  i  is  the  rank  and  N  the  sample  size. 

3.  Plot  the  probability  vs  the  value  on  probability  paper.  For  computer  made 

plots  this  may  require  computing  F(P)  to  determine  the  ordinates. 

4.  Optionally  color  the  points  according  to  attribute  to  make  apparent  any 

systematics  in  the  data. 

5.  Optionally  draw  the  one-sided  tolerance  limits. 

50.7  Some  Fine  Points.  The  quantity  i/(N+l)  derives  from  order  statistics. 
When  N  values  are  drawn  from  a  uniform  distribution  from  0  to  1  and  ranked,  the 
average  value  of  the  ith  ranked  point  is  i/(N+l).  For  normal  distributions  some 
prefer  to  use  (i-0. 3)/(N+0.4)  as  a  better  approximation  for  the  average  positions 
of  ranked  points.  The  difference  is  usually  of  no  significance.  There  are  some 
cases  in  the  literature  where  probability  is  calculated  from  i/N  but  this  is  an 
erroneous  procedure  except  in  special  cases  concerning  truncated  distributions. 

60  INTERPRETING  PLOTS 

60.1  Scrutinizing  probability  plots  requires  a  fair  amount  of  experience 
since  there  are  many  ways  in  which  data  can  go  wrong  and  many  situations  where 
good  data  may  be  rejected  because  of  subjective  conclusions.  This  section  will 
attempt  to  provide  some  feel  for  interpreting  probability  plots.  We  will  display 
t3rpical  plots,  strange  looking  plots,  examples  of  problematic  data  and  an  example 
of  a  step-stress  measurement. 

60.2  Intepreting  Typical  Plots.  To  obtain  typical  plots  on  normal 
probability  paper,  simulated  data  sets  consisting  of  50  points  were  generated 
from  a  normal  distribution.  FIGURE  II— 4  illustrates  an  ideal  case  where  about 
5  points  are  above  the  90%  confidence  line  and  about  5  points  are  below  the  10% 
confidence  line.  FIGURE  II-5  shows  a  better  than  usual  fit  where  many  of  the 
points  come  close  the  confidence  lines,  but  none  are  above  the  90%  confidence 
line  and  only  two  are  below  the  10%  confidence  line.  Such  fits  are  not  unusual. 
However,  if  the  data  fit  a  straight  line  too  well  and  none  of  the  measurements 
come  close  to  the  90%  and  10%  confidence  lines  then  non-random  sampling  is 
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indicated.  FIGURE  II~6  is  an  example  of  a  very  bad  fit  (the  worst  out  of  2000 
simulated  data  sets) .  Note  that  points  which  exceed  the  confidence  bounds  tend 
to  group  together  because  the  ranking  of  points  assures  a  correlation  between 
adjacent  points.  Probability  plots  are  not  tvntcal  regression  plots. 


StfIFLE  SIZE  SB 


ARBITRARy  UNITS 


FIGURE  II-4.  Ideal  normal  probability  plot  with  50  points  —  simulated 

data. 
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SAdPLE  SIZE  58  8  Oi^ER  98^  CONFIDENCE 


FIGURE  II-5.  Normal  probability  plot  of  50  points  with  an  overly  good 
butstill  acceptable  -  simulated  data. 
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SAMPLE  SIZE  50  15  POINTS  OVER  CONFIDENCE 


FIGURE  II-6.  Normal  probability  plot  of  50  points  with  an  exceptionally 

bad  fit  simulated  data. 


60.2.1  Deviations  from  A  Straight  Line,  The  eye  is  a  fair  indicator  of 
when  the  plot  deviates  enough  from  a  straight  line  to  suspect  that  a  GOF 
(Goodness  of  Fit)  test  is  needed.  Even  when  deviations  exist,  the  assumed 
distribution  may  be  an  acceptable  approximation  if  you  restrict  yourself  to 
certain  probability  ranges  and/or  do  "sample  size"  corrections  (discussed  later)  . 

60.3  Actual  Case  Histories  and  Their  Interpretation.  To  obtain  a  further 
feel  for  problems  which  will  arise  in  practice,  some  actual  situations  are 
presented  and  discussed. 

60.3.1  Systematic  Effects  And  Inhomogeneous  Lots.  TABLE  II-2  shows  a 
voltage  shift  for  20  ICs  (integrated  circuits)  irradiated  at  400  KRad.  There 
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were  5  ICs  (indicated  by  attributes  R1  through  R5)  each  having  4  nomina 
Identical  circuits  (indicated  by  attributes  Cl  through  C4) . 


lly 


TABLE  II-2.  Voltage  shift  at  400  Krad. 


UNRANKED 

DATA 

RANKED 

DATA 

VALUE  ATTRIBUTE 

RANK  VALUE 

ATTRIBUTE 

4.60, 

RlCl 

1  -5.70, 

R4C4 

4.31. 

R1C2 

2  -5.17, 

R2C3 

-2.00, 

R1C3 

3  -4.95, 

R4C3 

-2.13, 

R1C4 

4  -4.81, 

R2C4 

7.71. 

R2C1 

5  -2.62. 

R5C3 

7.67. 

R2C2 

6  -2.13, 

R1C4 

-5.17. 

R2C3 

7  -2.00, 

R1C3 

-4.81, 

R2C4 

8  -1.95, 

R5C4 

4.79, 

R3C1 

9  -1.80, 

R3C4 

3.76, 

R3C2 

10  -0.98, 

R3C3 

-0.98, 

R3C3 

11  3.76, 

R3C2 

-1.80, 

R3C4 

12  4.31, 

R1C2 

7.89, 

R4C1 

13  4.60, 

RlCl 

7.58, 

R4C2 

14  4.73, 

R5C2 

-4.95, 

R4C3 

15  4.79, 

R3C1 

-5.70, 

R4C4 

16  5.08, 

R5C1 

5.08, 

R5C1 

17  7.58, 

R4C2 

4.73, 

R5C2 

18  7.67, 

R2C2 

-2.62, 

R5C3 

19  7.71, 

R2C1 

-1.95, 

R5C4 

20  7.89, 

R4C1 

60 -3. 1.1  Example  of  inhomogeneous  data.  FIGURE  II-7  shows  the  resulting 
cumulative  probability  plot  on  normal  probability  paper  with  the  90%  and  10% 
confidence  lines.  It  is  clearly  not  a  typical  normal  distribution.  A  goodness 
of  fit  test  (to  be  discussed  later)  indicates  only  a  0,1%  confidence  that  this 
data  could  result  from  a  true  normal  distribution. 
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FIGURE  II-7.  Normal  probability  plot  of  actual  data  that  exhibited 

systematic  effects. 


60.3,1.1.1  Seeking  the  causes  of  inhomogeneltv  in  the  example.  A  study  of 
the  plot  shows  that  the  ICs  must  have  come  from  two  distinct  batches  (one  batch 
being  ICs  Rl,  R3,  R5  and  the  other  being  R2,  R4)  .  In  addition,  the  response  of 
the  circuits  varies  systematically  according  to  which  side  of  the  die  the  circuit 
was  fotind  (one  side  being  circuits  Cl  and  C2  and  the  other  being  C3  and  C4) . 
Thus,  any  circuit  belongs  to  one  of  4  groups  as  illustrated  in  FIGURE  II-8.  This 
figure  is  the  plot  of  FIGURE  II— 7  without  confidence  lines  but  with  the  points 
"colored"  to  show  how  each  group  corresponds  to  a  unique  set  of  attributes. 
Systematic  effects  are  present,  this  is  an  inhomogeneous  lot  and  sample  size 
corrections  (Section  70.3  and  DNA  documents  referenced  in  Section  2.1.1  of  the 
text)  indicate  an  effective  sample  size  of  6  which  is  not  in  statistical 
disagreement  with  the  fact  that  the  data  separates  into  4  distinct  groups.  Any 


74 


APPENDIX  II 


treatment  of  this  data  should  use  an  effective  sample  size  between  4  and  6  and 
not  estimate  probabilities  greater  than  about  84%.  If  the  system  survivability 
were  to  be  calculated,  the  specific  buy  of  parts  (how  many  batches)  would  be 
relevant  information.  Often  it  is  not  possible  to  find  a  set  of  attributes  which 
fit  some  obvious  clumping  of  points.  Nevertheless,  it  would  remain  very  likely 
that  systematics  are  a  serious  perturbation  and  sample  size  corrections  would  be 
appropriate  as  well  as  estimates  of  a  maximum  reliable  survival  probability  which 
could  be  estimated  from  that  effective  sample  size.  A  more  rigorous  calculation 
than  that  of  MIL— HDBK— 280  (See  Section  2.1.1  of  the  text)  is  not  yet  available 
in  usable  form. 


FIGURE  II-8.  Plot  of  figure  II-7  with  confidence  lines  omitted  and  points 

"colored" . 
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60.3.1.2  Subtle  Systematic  Effects.  We  have  seen  that  a  typical  signature 
of  systematic  effects  is  a  grouping  of  points  by  attribute  with  clear  gaps 
between  the  different  groups.  In  more  subtle  cases,  the  groups  may  not  clearly 
separate  and  the  plot  may  roughly  resemble  a  straight  line.  Judgment  will  be 
required  about  the  presence  of  systematics  and  inhomogeneities. 

60.3.2  Possible  Outlier  and  Possible  Step-Stress  Analysis  Required. 
Probability  plots  can  sometimes  be  useful  in  detecting  outliers  by  examining 
whether  the  points  at  the  extremes  of  the  plot  are  reasonably  near  a  straight 
line  fit  to  the  points.  The  following  example  will  serve  the  double  purpose  of 
illustrating  a  possible  outlier  and  illustrating  a  step-stress  measurement. 

60.3.2,1  Approximate  lognormal  distribution  for  this  example.  The 
distribution  of  stress  to  failure  was  found  from  previous  experience  to  be 
approximately  lognormal.  Accordingly,  the  logarithms  of  the  upper  and  lower 
bounds  corresponding  to  each  step-stress  measurement  were  taken  and  the  stresses 
to  failure  were  expressed  in  terms  of  an  average  logarithm  of  stress  to  failure 
and  a  spread  in  the  logarithm  between  the  upper  and  lower  bounds  and  the 
logarithms  were  treated  as  a  step  stress  measurement  for  a  normal  distribution. 
The  points  were  ranked  by  their  average  logarithms.  FIGURE  II-9,  the  resulting 
plot,  shows  that  step-stress  analysis  (See  Namenson  1984,  "Statistical  Analysis 
of  Step  Stress  Measurements  in  Hardness  Assurance,”  referenced  in  Section  2.2  of 
the  text)  might  be  required. 
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POSSIBLE  OimyiNG  POIKT 
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FIGURE  II-9.  Double  example  of  possible  outlier  and  step— stress  data 

actual  data. 


60,3.2.2  Possible  outlier  in  this  example.  The  point  corresponding  to  the 
highest  stress  to  failure  seems  too  far  off  the  curve  (only  its  lower  bound  is 
almost  acceptable)  hinting  that  it  may  be  an  outlier.  Further  investigation 
revealed  that  In  that  one  case,  the  stress  was  applied  using  a  different  geometry 
than  for  the  other  cases  giving  that  point  an  attribute  which  is  not  shared  by 
any  of  the  other  points,  its  distance  from  the  curve  and  its  unique  attribute 
both  contribute  to  a  judgmental  decision  that  the  point  is  an  outlier. 


60.3.3  Lot-to-Lot  Variations  and  Within  Lot  Variations.  Section  5. 2 .4 .4. 5 
of  the  main  text  states  that  when  many  lots  are  involved,  the  type  of  analysis 
which  is  performed  will  often  depend  on  the  relative  magnitudes  of  the  lot-to-loc 
variations  and  the  within  lot  variations.  In  general,  this  information  must  be 
extracted  from  the  data.  The  example  in  Section  60.3.1  of  this  appendix  further 
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WITHIM  LOT  I  LOT  TO  LOT  UftJlIfiTIONS  CONFIDENCE  LINES 


FIGURE  II-IO.  Within—lot  varlaClons  exceed  lot— to— lot  variations. 


indicates  that  sometloes  a  single  lot  does  not  reflect  a  homogeneous  population 
The  lognormal  probability  plot  of  FIGURE  II-IO  shows  the  opposite  case  where 
three  different  lots  were  mixed  together  to  produced  data  which  is  consistent 
with  a  single  lot.  No  dotibt,  if  a  large  amount  of  additional  data  were  obtained, 
systematic  lot— to— lot  differences  would  emerge  but  for  the  data  on  hand,  a 
single-lot  analysis  is  adequate  with  perhaps  a  sample  size  correction  for  added 
precision  and  with  limitations  on  the  maximum  survival  probability  that  can  be 
quoted. 

60.3.4  Approximate  Distributions .  In  practice,  there  is  often  little  or 
no  theory  about  the  probability  distribution  governing  part  parameters  and  stress 
to  failure.  Analysis  usually  proceeds  on  the  basis  sometimes  limited  experience. 
It  is,  therefore,  prudent  not  to  extrapolate  to  very  high  probabilities  on  the 
basis  of  a  small  amount  of  data.  FIGURE  II-ll,  which  will  be  explained  more 
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SM1P1E  SIZES:  20  50  108 

0.9997 
0.9990 

0.9970 
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^  8.7500 

n  0.5000 
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mm 

^  8.1000 
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0.8010 
8.0003 


FIGURE  II-ll.  Interpreting  gof  tests  using  n-3  equal-probability  zones 
where  n  is  sample  size.  Plot  also  shows  how  deviations  from  normal  form  may 
not  become  apparent  tmtil  a  sufficiently  large  sample  size  is  taken. 


completely  in  Section  70.2,  as  an  example  of  computer  generated  data  from 
simulations  of  chi-squared  goodness-of-flt  tests.  Each  data  point  represents  a 
simulation.  The  resulting  simulated  data  turned  out  to  be  quantized  at  specific 
values  (a  small  consideration)  and  to  deviate  from  a  nomal  distributions  at  the 
high  extreme.  Only  after  performing  thousands  of  simulations  did  the  deviations 
became  apparent  and  it  would  have  been  unwise  to  99.9%  probabilities  on  the  basis 
of  fewer  than  the  order  of  1000  samples.  As  a  general  rule,  unless  there  is 
strong  theoretical  or  empirical  evidence  to  justify  the  assumed  probability 
distribution,  when  probabilities  and  confidences  are  quoted,  the  probabilities 
should  not  be  less  than  the  order  of  l/N.^^  or  exceed  about  l-l/N,;^  where  N.^^  is 
the  effective  number  of  tested  parts. 
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70  Advanced  Procedures . 

70.1  Further  examination  of  reasonability  of  data.  We  have  seen  that 
constructing  probability  plots  is  a  first  step  in  evaluating  data.  The  plots  can 
also  be  used  to  get  estimates  of  relative  within-lot  and  lot-to-lot  variations 
and  sometimes  effective  sample  sizes.  Though  texts  on  probability  theory  have 
tests  for  specific  problems,  they  cannot  anticipate  every  complication  that  might 
arise  and  there  is  no  replacement  for  examining  probability  plots.  In  what 
follows  we  will  nevertheless  suggest,  for  the  reader  with  some  familiarity  with 
probability  theory,  some  specific  tests  on  the  reasonability  of  data. 

70.2  GOF  (Goodness-of-Flt)  Tests.  GOF  tests  are  described  in  almost  every 
text  on  probability  theory  (e.g.  Natrella^.  Usually,  a  proper  test  requires  that 
the  variable  space  from  minus  infinity  to  plus  infinity  be  partitioned  into  at 
least  5  contiguous  zones  and  with  at  least  5  devices  in  each  zone.  However,  when 
there  is  a  small  number  of  parts ,  these  requirements  may  make  the  usual  GOF  test 
impossible  and/or  very  insensitive  to  glaring  problems.  It  was  found,  however, 
by  Monte  Carlo  simulation  that  the  test  seems  to  work  if  the  number  of  zones  is 
as  large  as  N-3  even  though  this  will  mean  that  many  zones  will  have  less  than 
5  parts.  FIGURE  II-ll  shows  the  distribution  of  chi-squared  values  for  sample 
sizes  of  20,  50  and  100  when  the  range  of  the  normally  distributed  variable  from 
minus  infinity  to  plus  infinity  divided  into  N-3  approximately  zones  of 
approximately  equal  probability.  The  "staircase"  appearance  of  the  distribution 
of  SQR[  2(CHISQ))  ]  derives  from  the  fact  that  the  values  of  chi-squared  can  take 
on  only  discrete  values.  Evidently,  the  chi-squared  GOF  test  works  after  a 
fashion,  even  when  the  general  rules  for  its  validity  do  not  hold.  The  curves 
of  FIGURE  II-ll  may  be  used  to  evaluate  the  results  of  a  chi-sqxiared  GOF  test. 
(Note  that  the  number  of  degrees  of  freedom  is  N-6  when  the  test  is  done  with  N-3 
zones  so  clearly  this  test  cannon  be  done  with  7  or  fewer  parts.)  As  applied  to 
the  example  of  Section  60.3.1,  the  chi-squared  was  44.5  yielding  a  0.03% 
confidence  that  the  data  was  drawn  from  a  homogeneotis  normal  population. 

70.3  Sanmia  Size  Corrections.  For  this  method  to  work,  there  must  be 
approximately  the  same  nuinber  of  parts  in  each  "sublot"  of  an  inhomogeneous 
inspection  lot.  If  the  region  of  minus  infinity  to  plus  infinity  is  divided  into 
a  recommended  number  of  N— 3  equal  probability  zones,  the  effective  sample  size, 
N,e£,  may  be  calculated  from, 

N,£t-N  *  (N  -  6.5)  /  (Actual  CHISQ) 

As  applied  to  the  example  of  Section  60.3.1  with  an  actvial  sample  size  of 
20,  the  chi-squared  of  44.5  gives  an  effective  sample  size  of  6. 

70.4  Use  of  Confidence  lines.  If  confidence  lines  representing  one-sided 
tolerance  limits  are  placed  on  a  probability  plot,  the  number  of  data  items  to 
the  left  of  the  line  is  a  rough  test  of  whether  there  is  anything  suspicious  in 
the  data.  On  the  average,  fraction  C  of  the  data  items  should  lie  to  the  left 
of  the  C  confidence  line.  However,  there  are  wide  variations  about  the  average. 
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COtlPMISON  OF  SIV1PLI  SIZE  OF  28  WITH  SAfIFLE  SIZE 


FIGURE  11-12.  Expected  fraction  of  devices  which  exceed  90%  confidence 

line.  Sample  sizes  of  20  and  100. 


Since  an  analytic  calculation  of  exactly  how  this  varies,  a  Monte  Carlo 
simulation  was  performed.  The  probability  that  a  given  fraction  of  data  items 
to  the  left  of  the  confidence  line  seems  to  be  independent  of  the  sample 
size  as  illustrated  by  the  histogram  of  FIGURE  11-12  for  90%  confidence  lines. 
We  can  conclude  that  if  fewer  than  60%  of  the  points  lie  to  the  left  of  a  90% 
confidence  line,  for  example,  then  the  data  is  suspicious.  FIGURE  11-13  is  a 
similar  plot  for  10%  confidence  lines. 
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BENEFITS  OF  OVERTESTING 


There  are  significant  benefits  to  testing  at  a  greater  radiation 
level  than  the  specified  radiation  dose  level  (SRDL)  that  a  system 
must  survive.  In  brief,  for  a  given  required  part  survival 
probability  and  confidence  level  at  the  SRDL  level,  an  overtest  at 
X  times  SRDL  will  require  a  smaller  sample  size  than  the  same  test 
performed  at  SRDL.  Thus,  the  costs  of  testing  in  terms  of  sample 
device  costs  and  in  terms  of  the  shorter  time  required  to  do  the 
tests  on  a  reduced  sample  size  can  be  substantial.  In  addition, 
overstress  testing  ensures  that  abrupt  failures  will  not  occur 
right  at  SRDL.  It  is  also  true  that,  for  a  given  sample  size,  an 
overtest  at  X  *  SRDL  will  provide  an  increased  part  survival 
probability  at  SRDL.  This  latter  case  is  discussed  in  the 
following  paragraphs. 

It  is  straightforward  to  calculate  the  part  survival  probability  at 
SRDL  from  radiation  response  data  obtained  at  X  *  SRDL.  The 
procedure  is  to  calculate  an  end  point  limit  (RHEPL)  or  a  derated 
parameter  limit  (DPL)  (see  section  5. 2. 3. 6)  from  the  X  *  SRDL  data 
and  then  to  use  that  DPL  for  the  hardened  circuit  design  at  SRDL. 
Using  the  derated  parameter  limit  obtained  from  X  *  SRDL  data 
corresponds  to  having  an  increased  part  survival  probability,  P,  at 
SRDL. 

The  following  discussion,  based  on  lebo  data,  illustrates  how  the 
new  survival  probability,  P,  increases  at  SRDL  when  post¬ 
irradiation  measurements  are  made  at  X  *  SRDL  and  SRDL.  Assume  , 
for  this  example,  the  following  conditions: 

SRDL  -  10  krads(Si) 

Overtest  is  at  X  =  5,  so  X  *  SRDL  =  50  krads(Si) 

Sample  size  n  «  5 

Confidence  level  C  =  0.9  at  50  krads(Si) 

Part  survival  probability  P  =  0.95  at  50  krads(Si) 

For  these  values  of  n,  C,  and  P,  the  one-sided  tolerance  limit 
value,  KTL(5,0,9,095)  =  3.40. 

Assume  that  the  measured  radiation  response  data  is: 

MEAN (lebo (50  krads(Si)))  =  33.22  nA 
STDEV(Iebo(50  krads(Si)))  =  5.876  nA 
MEAN (lebo (10  krads)Si)))  »  13.88  nA 
STDEV(Iebo(10  krads(Si)))  =  1.580  nA 


The  derated  parameter  limit,  DPL,  at  50  )crads(Si)  is  then: 

DPL(50  krads(Si))  =  33.22  +  3.40  *  5.876  =  53.198 
and  this  DPL  is  to  be  used  at  10  krads(Si).  Thus 

53.198  =  13.88  +  KTL(5,  0.9,  ?)  *  1.580 

and  therefore  KTL(5,0.9,?)  must  be  24.88.  To  obtain  this  KTL,  the 
part  survival  probability  must  equal  0.999.  Thus,  in  this  example, 
the  part  survival  probability  has  increased  from  0.95  at  50 
krads(Si)  to  0.999  at  10  krads(Si). 

A  further  example  of  the  benefits  of  overtesting  is  shown  in  Table 
III-l  (ref.  "Hardness  Assurance  and  Overtesting,”  A.  I.  Namenson, 
IEEE  Trans,  on  Nucl.  Sci.,  NS  -29,  1821,  (1982)).  The  table  shows 
how  overtesting  can  reduce  the  number  of  parts  that  are  subjected 
to  an  LTPD  test  with  no  failures  as  the  acceptance  criterion.  We 
assume  that  the  parameters  are  lognormally  distributed,  that  the 
required  part  survival  probability  is  0.999,  the  maximum  standard 
deviation  in  the  logarithms  is  0.83,  and  the  confidence  level  is  to 
be  0.90.  Table  III-l  lists  the  overtest  level,  sample  size  (the 
number  to  be  tested  at  the  overtest  level  with  no  failures 
allowed) ,  and  the  LTPD  (lot  tolerance  percent  defective) 
probability.  The  differences  are  striking.  To  get  the  desired 
survival  probability  of  0.999  and  the  desired  confidence  level  of 
0.9,  an  LTPD  test  at  SRDL  requires  2302  parts  to  pass  without  a 
single  failure,  while  an  LTPD  test  at  10  *  SRDL  only  requires  5 
parts . 


Table  Overtest  level,  sample  size  with  no  measured 
failures,  and  LTPD  probability  for  a  survivability  of  0.999, 
confidence  level  of  0.9,  with  a  maximum  likely  standard  deviation 
in  the  logarithms  of  0.83. 


OVERTEST 

LEVEL 

lOX 

5X 

4X 

3X 

2X 

IX 

SAMPLE 

SIZE 

5 

17 

29 

60 

191 

2302 

LTPD 

PROBABILITY 

0.631 

0.873 

0.922 

0.962 

0.988 

0.999 

r 
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